2024-09-21T11:36:43.212 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T11:36:43.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T11:36:43.340 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877 branch: wip-guits-main-2024-09-21-0813 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7913877' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-21-0813 ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - CEPHADM_STRAY_DAEMON log-only-match: - CEPHADM_ sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-main-2024-09-21-0813 sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 3041 sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-21-0813 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_5c1cdc3c470dae78e91a0ed2bcf13509a046d301/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBVodlidCUDtIsrhnlHEwq1lK5JbXP7bvE6gHbrzLQx1YqmsRHSplfUEWPOAieZKCjr4KhD9FNF/7loimsNKIJQ= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDIPEfeVi6FeiwxNXwa8uw1NOAP1lgC85OP9kqF8bfd/onRIUGStv8/ruQ5lqRcJunA4bo3o7vGV/qqbr47odgo= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_10:07:29 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T11:36:43.340 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_5c1cdc3c470dae78e91a0ed2bcf13509a046d301/qa; will attempt to use it 2024-09-21T11:36:43.341 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_5c1cdc3c470dae78e91a0ed2bcf13509a046d301/qa/tasks 2024-09-21T11:36:43.341 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T11:36:43.343 INFO:teuthology.task.internal:Checking packages... 2024-09-21T11:36:43.369 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5c1cdc3c470dae78e91a0ed2bcf13509a046d301' 2024-09-21T11:36:43.369 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T11:36:43.369 INFO:teuthology.packaging:ref: None 2024-09-21T11:36:43.369 INFO:teuthology.packaging:tag: None 2024-09-21T11:36:43.369 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-21-0813 2024-09-21T11:36:43.370 INFO:teuthology.packaging:sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:36:43.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-21-0813 2024-09-21T11:36:43.511 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5103.g5c1cdc3c 2024-09-21T11:36:43.513 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T11:36:43.522 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T11:36:43.523 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T11:36:43.536 INFO:teuthology.task.internal:Saving configuration 2024-09-21T11:36:43.549 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T11:36:43.558 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T11:36:43.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 11:31:32.945791', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBVodlidCUDtIsrhnlHEwq1lK5JbXP7bvE6gHbrzLQx1YqmsRHSplfUEWPOAieZKCjr4KhD9FNF/7loimsNKIJQ='} 2024-09-21T11:36:43.612 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 11:31:32.944324', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDIPEfeVi6FeiwxNXwa8uw1NOAP1lgC85OP9kqF8bfd/onRIUGStv8/ruQ5lqRcJunA4bo3o7vGV/qqbr47odgo='} 2024-09-21T11:36:43.612 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T11:36:43.658 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-21T11:36:43.658 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-21T11:36:43.658 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T11:36:43.747 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0e329f0280>, signals=[15]) 2024-09-21T11:36:43.747 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T11:36:43.760 INFO:teuthology.task.internal:Opening connections... 2024-09-21T11:36:43.761 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T11:36:43.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:36:43.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T11:36:43.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:36:43.915 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T11:36:43.921 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-21T11:36:43.938 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-21T11:36:43.939 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T11:36:43.996 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-21T11:36:43.997 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-21T11:36:43.997 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T11:36:43.997 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-21T11:36:43.997 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T11:36:43.997 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T11:36:43.997 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T11:36:43.997 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-21T11:36:44.020 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-21T11:36:44.039 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-21T11:36:44.039 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-21T11:36:44.097 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T11:36:44.098 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T11:36:44.098 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T11:36:44.098 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-21T11:36:44.120 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T11:36:44.129 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T11:36:44.135 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T11:36:44.135 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-21T11:36:44.137 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-21T11:36:44.154 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T11:36:44.160 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T11:36:44.160 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-21T11:36:44.194 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-21T11:36:44.287 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T11:36:44.340 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T11:36:44.354 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T11:36:44.354 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T11:36:44.355 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T11:36:44.355 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T11:36:44.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:36:44.372 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-21T11:36:44.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:36:44.373 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-09-21T11:36:44.430 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-503.el9.x86_64 2024-09-21T11:36:44.430 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-503.el9.x86_64 2024-09-21T11:36:44.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-09-21T11:36:44.431 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-503.el9.x86_64 2024-09-21T11:36:44.432 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-503.el9.x86_64 2024-09-21T11:36:44.432 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-09-21T11:36:46.717 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-21T11:36:46.755 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T11:36:48.501 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-21T11:36:48.573 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T11:36:52.777 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T11:36:54.252 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-21T11:36:54.881 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-21T11:36:55.331 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T11:36:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-21T11:36:57.136 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T11:37:00.829 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-21T11:37:02.591 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-21T11:37:02.693 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T11:37:02.694 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T11:37:02.793 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T11:37:02.803 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 79 M 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 126 M 2024-09-21T11:37:02.804 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:37:02.913 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 405 kB/s | 36 kB 00:00 2024-09-21T11:37:03.205 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-09-21T11:37:03.497 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 49 MB/s | 29 MB 00:00 2024-09-21T11:37:04.214 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 25 MB/s | 35 MB 00:01 2024-09-21T11:37:04.214 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:37:04.214 INFO:teuthology.orchestra.run.smithi193.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-21T11:37:04.514 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T11:37:04.514 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T11:37:04.606 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:37:04.611 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:37:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-21T11:37:04.611 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:37:04.611 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-21T11:37:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:37:04.613 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 79 M 2024-09-21T11:37:04.613 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 126 M 2024-09-21T11:37:04.613 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:37:04.834 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 349 kB/s | 36 kB 00:00 2024-09-21T11:37:04.900 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:37:05.262 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:37:05.262 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:37:05.301 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 61 MB/s | 29 MB 00:00 2024-09-21T11:37:05.560 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-21T11:37:06.118 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 25 MB/s | 35 MB 00:01 2024-09-21T11:37:06.118 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:37:06.118 INFO:teuthology.orchestra.run.smithi107.stdout:Total 52 MB/s | 79 MB 00:01 2024-09-21T11:37:06.224 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:37:06.224 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:37:06.805 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:37:07.165 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:37:07.166 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:37:08.120 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:37:08.120 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:37:08.654 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:37:09.398 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:37:09.497 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:37:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:37:11.256 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:37:11.343 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:37:12.240 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:37:12.392 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:37:14.098 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:37:14.245 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:37:17.249 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:37:17.343 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:37:19.084 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:37:19.231 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:37:22.206 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:37:24.060 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:00.487 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:00.527 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:00.842 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:00.843 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:38:00.843 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:38:00.843 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:38:01.651 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:38:02.009 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:38:02.095 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:38:02.096 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:38:02.097 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T11:38:02.097 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T11:38:02.097 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T11:38:02.097 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T11:38:02.097 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi193.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T11:38:02.097 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:38:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:02.155 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:38:02.156 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:38:02.156 DEBUG:teuthology.orchestra.run.smithi193:> sudo rpm -qi grub2-tools 2024-09-21T11:38:02.182 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:02.191 INFO:teuthology.orchestra.run.smithi193.stdout:Name : grub2-tools 2024-09-21T11:38:02.191 INFO:teuthology.orchestra.run.smithi193.stdout:Epoch : 1 2024-09-21T11:38:02.191 INFO:teuthology.orchestra.run.smithi193.stdout:Version : 2.06 2024-09-21T11:38:02.191 INFO:teuthology.orchestra.run.smithi193.stdout:Release : 82.el9 2024-09-21T11:38:02.192 INFO:teuthology.orchestra.run.smithi193.stdout:Architecture: x86_64 2024-09-21T11:38:02.192 INFO:teuthology.orchestra.run.smithi193.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T11:38:02.192 INFO:teuthology.orchestra.run.smithi193.stdout:Group : Unspecified 2024-09-21T11:38:02.192 INFO:teuthology.orchestra.run.smithi193.stdout:Size : 8274670 2024-09-21T11:38:02.192 INFO:teuthology.orchestra.run.smithi193.stdout:License : GPLv3+ 2024-09-21T11:38:02.192 INFO:teuthology.orchestra.run.smithi193.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T11:38:02.193 INFO:teuthology.orchestra.run.smithi193.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T11:38:02.193 INFO:teuthology.orchestra.run.smithi193.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T11:38:02.193 INFO:teuthology.orchestra.run.smithi193.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T11:38:02.193 INFO:teuthology.orchestra.run.smithi193.stdout:Packager : builder@centos.org 2024-09-21T11:38:02.193 INFO:teuthology.orchestra.run.smithi193.stdout:Vendor : CentOS 2024-09-21T11:38:02.193 INFO:teuthology.orchestra.run.smithi193.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T11:38:02.194 INFO:teuthology.orchestra.run.smithi193.stdout:Summary : Support tools for GRUB. 2024-09-21T11:38:02.194 INFO:teuthology.orchestra.run.smithi193.stdout:Description : 2024-09-21T11:38:02.194 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:38:02.194 INFO:teuthology.orchestra.run.smithi193.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T11:38:02.194 INFO:teuthology.orchestra.run.smithi193.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T11:38:02.194 INFO:teuthology.orchestra.run.smithi193.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T11:38:02.195 INFO:teuthology.orchestra.run.smithi193.stdout:hardware devices. 2024-09-21T11:38:02.195 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:38:02.195 INFO:teuthology.orchestra.run.smithi193.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T11:38:02.196 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T11:38:02.196 INFO:teuthology.task.kernel:Updating grub on smithi193 to boot 5.14.0-508.el9.x86_64 2024-09-21T11:38:02.196 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T11:38:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:38:02.484 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:38:02.485 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:38:02.860 INFO:teuthology.orchestra.run.smithi193.stderr:Generating grub configuration file ... 2024-09-21T11:38:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:38:03.357 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:38:03.710 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:38:03.785 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:38:03.785 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:38:03.785 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T11:38:03.785 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T11:38:03.785 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T11:38:03.785 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T11:38:03.785 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T11:38:03.785 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:38:03.844 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:38:03.844 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:38:03.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 82.el9 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8274670 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-09-21T11:38:03.876 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-09-21T11:38:03.877 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:38:03.878 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T11:38:03.878 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T11:38:03.878 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 5.14.0-508.el9.x86_64 2024-09-21T11:38:03.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T11:38:04.210 INFO:teuthology.orchestra.run.smithi193.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T11:38:04.238 INFO:teuthology.orchestra.run.smithi193.stderr:done 2024-09-21T11:38:04.244 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-09-21T11:38:04.259 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.Fq7aSTZUex 2024-09-21T11:38:04.260 DEBUG:teuthology.orchestra.run.smithi193:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Fq7aSTZUex 2024-09-21T11:38:04.328 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.Fq7aSTZUex 2024-09-21T11:38:04.467 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.Fq7aSTZUex is 6KB 2024-09-21T11:38:04.515 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.Fq7aSTZUex 2024-09-21T11:38:04.530 DEBUG:teuthology.orchestra.run.smithi193:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T11:38:04.592 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T11:38:04.592 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T11:38:04.592 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T11:38:04.592 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T11:38:04.594 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T11:38:04.620 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-21T11:38:04.759 DEBUG:teuthology.orchestra.run.smithi193:> sudo shutdown -r now 2024-09-21T11:38:05.961 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T11:38:05.997 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-21T11:38:05.999 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-21T11:38:06.055 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.6SFmmM9cC9 2024-09-21T11:38:06.055 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6SFmmM9cC9 2024-09-21T11:38:06.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.6SFmmM9cC9 2024-09-21T11:38:06.271 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.6SFmmM9cC9 is 6KB 2024-09-21T11:38:06.320 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.6SFmmM9cC9 2024-09-21T11:38:06.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T11:38:06.398 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T11:38:06.399 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T11:38:06.399 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T11:38:06.399 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T11:38:06.400 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T11:38:06.551 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-21T11:38:34.790 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T11:38:34.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T11:38:34.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:38:36.556 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T11:38:36.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:38:36.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:38:53.295 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-09-21T11:38:55.091 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-21T11:39:02.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T11:39:02.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:39:04.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:39:04.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:39:05.359 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-09-21T11:39:07.151 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-21T11:39:17.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T11:39:17.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:39:19.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:39:19.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:39:37.587 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-21T11:39:49.222 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-21T11:39:49.594 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T11:39:49.594 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T11:39:49.594 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-09-21T11:39:49.608 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-508.el9.x86_64 2024-09-21T11:39:49.608 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T11:39:49.608 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T11:39:49.609 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T11:39:50.609 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T11:39:50.609 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T11:39:50.694 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-09-21T11:39:50.716 DEBUG:teuthology.parallel:result is None 2024-09-21T11:39:52.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:39:52.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:39:53.073 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-21T11:39:53.408 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:39:53.408 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T11:39:53.409 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-21T11:39:53.423 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-508.el9.x86_64 2024-09-21T11:39:53.423 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T11:39:53.423 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T11:39:53.423 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T11:39:54.425 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T11:39:54.425 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T11:39:54.486 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-09-21T11:39:54.515 DEBUG:teuthology.parallel:result is None 2024-09-21T11:39:54.515 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T11:39:54.522 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T11:39:54.522 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T11:39:54.524 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T11:39:54.541 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T11:39:54.547 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T11:39:54.560 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T11:39:54.560 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T11:39:54.579 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T11:39:54.608 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T11:39:54.614 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T11:39:54.615 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T11:39:54.635 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T11:39:54.659 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:39:54.669 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:39:54.672 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:39:54.683 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:39:54.684 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T11:39:54.691 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T11:39:54.692 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T11:39:54.714 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T11:39:54.750 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T11:39:54.762 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T11:39:54.762 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T11:39:54.778 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T11:39:54.806 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:39:54.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:39:54.915 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:39:54.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:39:55.038 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:39:55.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T11:39:55.103 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:39:55.144 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:39:55.194 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:39:55.275 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:39:55.316 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:39:55.316 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T11:39:55.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-21T11:39:55.384 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-21T11:39:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T11:39:55.473 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T11:39:55.986 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T11:39:55.994 INFO:teuthology.task.internal:Starting timer... 2024-09-21T11:39:55.994 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T11:39:56.005 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T11:39:56.013 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T11:39:56.013 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-21T11:39:56.082 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-21T11:39:56.084 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-21T11:39:56.140 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-21T11:39:56.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T11:39:56.142 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-21T11:39:56.174 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-21T11:39:56.174 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-21T11:39:56.200 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-21T11:39:56.200 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-09-21T11:39:56.200 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T11:39:56.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:39:56.240 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T11:39:56.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:39:56.266 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T11:39:56.266 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-21T11:39:56.311 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-21T11:39:56.342 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T11:39:56.356 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T11:39:56.359 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T11:39:56.465 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T11:39:57.562 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T11:39:57.597 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T11:43:45.099 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-09-21T11:43:45.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:43:45.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:43:45.180 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-21T11:43:45.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T11:43:45.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T11:43:45.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:43:45.356 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-21T11:43:45.439 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T11:43:45.439 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T11:43:45.449 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T11:43:45.449 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T11:43:45.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T11:43:45.452 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T11:43:45.452 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T11:43:45.482 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T11:43:45.498 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T11:43:45.510 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T11:43:45.524 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-21T11:43:45.525 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T11:43:45.536 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-21T11:43:45.550 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T11:43:45.551 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-21T11:43:45.562 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-21T11:43:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T11:43:45.577 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T11:43:45.592 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T11:43:45.623 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-21T11:43:45.627 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T11:43:45.627 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-21T11:43:45.627 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.627 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.627 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.627 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.652 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-21T11:43:45.656 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T11:43:45.656 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-21T11:43:45.656 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.656 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.656 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.656 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:43:45.656 INFO:teuthology.run_tasks:Running task pexec... 2024-09-21T11:43:45.665 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-21T11:43:45.666 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T11:43:45.666 DEBUG:teuthology.orchestra.run.smithi107:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-21T11:43:45.666 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T11:43:45.666 DEBUG:teuthology.orchestra.run.smithi193:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-21T11:43:46.019 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:43:46.021 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:43:46.021 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:Removing: 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:Removing unused dependencies: 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:Remove 2 Packages 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:Freed space: 5.5 M 2024-09-21T11:43:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:43:46.026 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:43:46.026 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:43:46.047 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:43:46.059 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:43:46.059 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-21T11:43:46.059 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:43:46.059 INFO:teuthology.orchestra.run.smithi193.stdout:Removing: 2024-09-21T11:43:46.059 INFO:teuthology.orchestra.run.smithi193.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout:Removing unused dependencies: 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout:Remove 2 Packages 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout:Freed space: 5.5 M 2024-09-21T11:43:46.060 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:43:46.064 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:43:46.064 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:43:46.090 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:43:46.090 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:43:46.129 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:43:46.130 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:43:46.233 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:43:46.276 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:43:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-21T11:43:46.364 INFO:teuthology.orchestra.run.smithi193.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-21T11:43:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:43:46.438 INFO:teuthology.orchestra.run.smithi193.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:43:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:43:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-21T11:43:46.933 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:43:46.933 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-21T11:43:47.150 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-21T11:43:47.150 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:47.151 INFO:teuthology.orchestra.run.smithi193.stdout:Removed: 2024-09-21T11:43:47.151 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:43:47.151 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:47.151 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:43:47.156 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-21T11:43:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:47.157 INFO:teuthology.orchestra.run.smithi107.stdout:Removed: 2024-09-21T11:43:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:43:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:47.157 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:43:47.988 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:43 ago on Sat 21 Sep 2024 11:43:04 AM UTC. 2024-09-21T11:43:47.995 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:52 ago on Sat 21 Sep 2024 11:42:55 AM UTC. 2024-09-21T11:43:48.124 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:43:48.124 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:48.125 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:43:48.126 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:43:48.126 INFO:teuthology.orchestra.run.smithi107.stdout:Install 7 Packages 2024-09-21T11:43:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:48.131 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-21T11:43:48.132 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:48.133 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:43:48.133 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:43:48.133 INFO:teuthology.orchestra.run.smithi193.stdout:Install 7 Packages 2024-09-21T11:43:48.133 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:48.135 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 5.2 M 2024-09-21T11:43:48.135 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 19 M 2024-09-21T11:43:48.135 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:43:48.144 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 5.2 M 2024-09-21T11:43:48.144 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 19 M 2024-09-21T11:43:48.144 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:43:48.389 INFO:teuthology.orchestra.run.smithi193.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 372 kB/s | 44 kB 00:00 2024-09-21T11:43:48.423 INFO:teuthology.orchestra.run.smithi193.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 691 kB/s | 106 kB 00:00 2024-09-21T11:43:48.460 INFO:teuthology.orchestra.run.smithi107.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 490 kB/s | 44 kB 00:00 2024-09-21T11:43:48.464 INFO:teuthology.orchestra.run.smithi193.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 952 kB/s | 72 kB 00:00 2024-09-21T11:43:48.486 INFO:teuthology.orchestra.run.smithi107.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 913 kB/s | 106 kB 00:00 2024-09-21T11:43:48.490 INFO:teuthology.orchestra.run.smithi193.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-21T11:43:48.523 INFO:teuthology.orchestra.run.smithi193.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.5 MB/s | 906 kB 00:00 2024-09-21T11:43:48.527 INFO:teuthology.orchestra.run.smithi107.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s | 72 kB 00:00 2024-09-21T11:43:48.553 INFO:teuthology.orchestra.run.smithi107.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-21T11:43:48.590 INFO:teuthology.orchestra.run.smithi193.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 6.5 MB/s | 837 kB 00:00 2024-09-21T11:43:48.595 INFO:teuthology.orchestra.run.smithi107.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.9 MB/s | 906 kB 00:00 2024-09-21T11:43:48.637 INFO:teuthology.orchestra.run.smithi107.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.5 MB/s | 837 kB 00:00 2024-09-21T11:43:49.266 INFO:teuthology.orchestra.run.smithi193.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 4.1 MB/s | 3.2 MB 00:00 2024-09-21T11:43:49.266 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:43:49.266 INFO:teuthology.orchestra.run.smithi193.stdout:Total 4.6 MB/s | 5.2 MB 00:01 2024-09-21T11:43:49.377 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:43:49.392 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:43:49.392 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:43:49.709 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:43:49.710 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:43:49.761 INFO:teuthology.orchestra.run.smithi107.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.6 MB/s | 3.2 MB 00:01 2024-09-21T11:43:49.762 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:43:49.762 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.2 MB/s | 5.2 MB 00:01 2024-09-21T11:43:49.880 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:43:49.895 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:43:49.895 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:43:50.179 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:43:50.208 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:43:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:43:50.276 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-21T11:43:50.319 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-21T11:43:50.348 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-21T11:43:50.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-21T11:43:50.588 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-21T11:43:50.681 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:43:50.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-21T11:43:50.811 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-21T11:43:50.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-21T11:43:51.022 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-21T11:43:51.042 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-21T11:43:51.049 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-21T11:43:51.083 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-21T11:43:51.167 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-21T11:43:51.228 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-21T11:43:51.529 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-21T11:43:51.563 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-21T11:43:51.677 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-21T11:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-21T11:43:52.212 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-21T11:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-21T11:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-21T11:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-21T11:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-21T11:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-21T11:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-21T11:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:43:52.716 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-21T11:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-21T11:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:43:53.104 DEBUG:teuthology.parallel:result is None 2024-09-21T11:43:53.508 DEBUG:teuthology.parallel:result is None 2024-09-21T11:43:53.508 INFO:teuthology.run_tasks:Running task install... 2024-09-21T11:43:53.517 DEBUG:teuthology.task.install:project ceph 2024-09-21T11:43:53.517 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'}} 2024-09-21T11:43:53.517 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'} 2024-09-21T11:43:53.517 INFO:teuthology.task.install:Using flavor: default 2024-09-21T11:43:53.523 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T11:43:53.523 INFO:teuthology.task.install:extra packages: [] 2024-09-21T11:43:53.523 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'tag': None, 'wait_for_package': False} 2024-09-21T11:43:53.523 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:43:53.525 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'tag': None, 'wait_for_package': False} 2024-09-21T11:43:53.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:43:53.666 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/ 2024-09-21T11:43:53.667 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5103.g5c1cdc3c 2024-09-21T11:43:53.670 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/ 2024-09-21T11:43:53.670 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5103.g5c1cdc3c 2024-09-21T11:43:53.775 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T11:43:53.775 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:43:53.775 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T11:43:53.783 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T11:43:53.783 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:43:53.783 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T11:43:53.802 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T11:43:53.802 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T11:43:53.814 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T11:43:53.814 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T11:43:53.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T11:43:53.883 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T11:43:53.945 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T11:43:53.961 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T11:43:54.010 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-21T11:43:54.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-21T11:43:54.028 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-09-21T11:43:54.029 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-09-21T11:43:54.255 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-21T11:43:54.260 INFO:teuthology.orchestra.run.smithi193.stdout:45 files removed 2024-09-21T11:43:54.284 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-09-21T11:43:54.288 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw 2024-09-21T11:43:54.842 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-21T11:43:54.849 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T11:43:55.134 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T11:43:55.141 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T11:43:55.409 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-21T11:43:55.416 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-21T11:43:57.201 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-21T11:43:57.375 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-21T11:43:58.877 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-21T11:44:02.951 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-21T11:44:03.261 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-21T11:44:05.696 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-21T11:44:06.593 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T11:44:08.476 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-21T11:44:09.919 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-21T11:44:11.015 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T11:44:12.081 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-21T11:44:14.321 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 11 M 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.5 M 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:44:14.323 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 22 M 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 25 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 755 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 509 k 2024-09-21T11:44:14.324 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 45 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 148 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 321 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 306 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 99 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T11:44:14.325 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Install 31 Packages 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-09-21T11:44:14.326 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:14.327 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 62 M 2024-09-21T11:44:14.327 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:44:15.283 INFO:teuthology.orchestra.run.smithi107.stdout:(1/33): ceph-base-19.3.0-5103.g5c1cdc3c.el9.x86 8.9 MB/s | 5.5 MB 00:00 2024-09-21T11:44:15.341 INFO:teuthology.orchestra.run.smithi107.stdout:(2/33): ceph-selinux-19.3.0-5103.g5c1cdc3c.el9. 431 kB/s | 25 kB 00:00 2024-09-21T11:44:15.475 INFO:teuthology.orchestra.run.smithi107.stdout:(3/33): ceph-radosgw-19.3.0-5103.g5c1cdc3c.el9. 13 MB/s | 11 MB 00:00 2024-09-21T11:44:15.517 INFO:teuthology.orchestra.run.smithi107.stdout:(4/33): libcephfs2-19.3.0-5103.g5c1cdc3c.el9.x8 4.2 MB/s | 755 kB 00:00 2024-09-21T11:44:15.542 INFO:teuthology.orchestra.run.smithi107.stdout:(5/33): libradosstriper1-19.3.0-5103.g5c1cdc3c. 7.4 MB/s | 509 kB 00:00 2024-09-21T11:44:15.584 INFO:teuthology.orchestra.run.smithi107.stdout:(6/33): python3-ceph-argparse-19.3.0-5103.g5c1c 1.1 MB/s | 45 kB 00:00 2024-09-21T11:44:15.684 INFO:teuthology.orchestra.run.smithi107.stdout:(7/33): python3-ceph-common-19.3.0-5103.g5c1cdc 1.5 MB/s | 148 kB 00:00 2024-09-21T11:44:15.935 INFO:teuthology.orchestra.run.smithi107.stdout:(8/33): ceph-common-19.3.0-5103.g5c1cdc3c.el9.x 17 MB/s | 22 MB 00:01 2024-09-21T11:44:15.960 INFO:teuthology.orchestra.run.smithi107.stdout:(9/33): python3-cephfs-19.3.0-5103.g5c1cdc3c.el 586 kB/s | 161 kB 00:00 2024-09-21T11:44:16.019 INFO:teuthology.orchestra.run.smithi107.stdout:(10/33): python3-rados-19.3.0-5103.g5c1cdc3c.el 3.8 MB/s | 321 kB 00:00 2024-09-21T11:44:16.086 INFO:teuthology.orchestra.run.smithi107.stdout:(11/33): librgw2-19.3.0-5103.g5c1cdc3c.el9.x86_ 9.7 MB/s | 5.5 MB 00:00 2024-09-21T11:44:16.111 INFO:teuthology.orchestra.run.smithi107.stdout:(12/33): python3-rgw-19.3.0-5103.g5c1cdc3c.el9. 1.1 MB/s | 99 kB 00:00 2024-09-21T11:44:16.145 INFO:teuthology.orchestra.run.smithi107.stdout:(13/33): python3-rbd-19.3.0-5103.g5c1cdc3c.el9. 1.6 MB/s | 306 kB 00:00 2024-09-21T11:44:16.287 INFO:teuthology.orchestra.run.smithi107.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 533 kB/s | 107 kB 00:00 2024-09-21T11:44:16.312 INFO:teuthology.orchestra.run.smithi107.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 851 kB/s | 171 kB 00:00 2024-09-21T11:44:16.337 INFO:teuthology.orchestra.run.smithi107.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-21T11:44:16.363 INFO:teuthology.orchestra.run.smithi107.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 736 kB/s | 160 kB 00:00 2024-09-21T11:44:16.404 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T11:44:16.413 INFO:teuthology.orchestra.run.smithi107.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 848 kB/s | 42 kB 00:00 2024-09-21T11:44:16.438 INFO:teuthology.orchestra.run.smithi107.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-21T11:44:16.463 INFO:teuthology.orchestra.run.smithi107.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T11:44:16.489 INFO:teuthology.orchestra.run.smithi107.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-09-21T11:44:16.514 INFO:teuthology.orchestra.run.smithi107.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-21T11:44:16.539 INFO:teuthology.orchestra.run.smithi107.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-21T11:44:16.598 INFO:teuthology.orchestra.run.smithi107.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-21T11:44:16.623 INFO:teuthology.orchestra.run.smithi107.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-21T11:44:16.648 INFO:teuthology.orchestra.run.smithi107.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 276 kB/s | 26 kB 00:00 2024-09-21T11:44:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T11:44:16.716 INFO:teuthology.orchestra.run.smithi107.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T11:44:16.758 INFO:teuthology.orchestra.run.smithi107.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T11:44:16.816 INFO:teuthology.orchestra.run.smithi107.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T11:44:16.883 INFO:teuthology.orchestra.run.smithi107.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T11:44:17.017 INFO:teuthology.orchestra.run.smithi107.stdout:(32/33): librados2-19.3.0-5103.g5c1cdc3c.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-21T11:44:17.076 INFO:teuthology.orchestra.run.smithi107.stdout:(33/33): librbd1-19.3.0-5103.g5c1cdc3c.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-21T11:44:17.081 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:44:17.081 INFO:teuthology.orchestra.run.smithi107.stdout:Total 22 MB/s | 62 MB 00:02 2024-09-21T11:44:17.197 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:44:17.234 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:44:17.234 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:44:17.743 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:44:17.744 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:44:18.641 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:44:18.643 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:18.643 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 11 M 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.5 M 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 22 M 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 25 k 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T11:44:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 755 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 509 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 45 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 148 k 2024-09-21T11:44:18.645 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 321 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 306 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 99 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:18.646 INFO:teuthology.orchestra.run.smithi193.stdout:Install 31 Packages 2024-09-21T11:44:18.647 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 2 Packages 2024-09-21T11:44:18.647 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:18.647 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 62 M 2024-09-21T11:44:18.647 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:44:19.207 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:44:19.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-21T11:44:19.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-21T11:44:19.339 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-21T11:44:19.366 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:44:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:44:19.543 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:44:19.572 INFO:teuthology.orchestra.run.smithi193.stdout:(1/33): ceph-base-19.3.0-5103.g5c1cdc3c.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-21T11:44:19.596 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:44:19.631 INFO:teuthology.orchestra.run.smithi193.stdout:(2/33): ceph-selinux-19.3.0-5103.g5c1cdc3c.el9. 432 kB/s | 25 kB 00:00 2024-09-21T11:44:19.731 INFO:teuthology.orchestra.run.smithi193.stdout:(3/33): libcephfs2-19.3.0-5103.g5c1cdc3c.el9.x8 7.4 MB/s | 755 kB 00:00 2024-09-21T11:44:19.848 INFO:teuthology.orchestra.run.smithi193.stdout:(4/33): ceph-radosgw-19.3.0-5103.g5c1cdc3c.el9. 14 MB/s | 11 MB 00:00 2024-09-21T11:44:19.873 INFO:teuthology.orchestra.run.smithi193.stdout:(5/33): libradosstriper1-19.3.0-5103.g5c1cdc3c. 3.5 MB/s | 509 kB 00:00 2024-09-21T11:44:19.932 INFO:teuthology.orchestra.run.smithi193.stdout:(6/33): python3-ceph-argparse-19.3.0-5103.g5c1c 773 kB/s | 45 kB 00:00 2024-09-21T11:44:19.942 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-21T11:44:20.024 INFO:teuthology.orchestra.run.smithi193.stdout:(7/33): python3-ceph-common-19.3.0-5103.g5c1cdc 1.6 MB/s | 148 kB 00:00 2024-09-21T11:44:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:44:20.107 INFO:teuthology.orchestra.run.smithi193.stdout:(8/33): librgw2-19.3.0-5103.g5c1cdc3c.el9.x86_6 21 MB/s | 5.5 MB 00:00 2024-09-21T11:44:20.133 INFO:teuthology.orchestra.run.smithi193.stdout:(9/33): python3-cephfs-19.3.0-5103.g5c1cdc3c.el 1.5 MB/s | 161 kB 00:00 2024-09-21T11:44:20.158 INFO:teuthology.orchestra.run.smithi193.stdout:(10/33): python3-rados-19.3.0-5103.g5c1cdc3c.el 6.3 MB/s | 321 kB 00:00 2024-09-21T11:44:20.163 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:44:20.192 INFO:teuthology.orchestra.run.smithi193.stdout:(11/33): python3-rbd-19.3.0-5103.g5c1cdc3c.el9. 5.1 MB/s | 306 kB 00:00 2024-09-21T11:44:20.217 INFO:teuthology.orchestra.run.smithi193.stdout:(12/33): python3-rgw-19.3.0-5103.g5c1cdc3c.el9. 1.6 MB/s | 99 kB 00:00 2024-09-21T11:44:20.253 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 8/35 2024-09-21T11:44:20.281 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:44:20.335 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:44:20.373 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-21T11:44:20.424 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 11/35 2024-09-21T11:44:20.426 INFO:teuthology.orchestra.run.smithi193.stdout:(13/33): ceph-common-19.3.0-5103.g5c1cdc3c.el9. 16 MB/s | 22 MB 00:01 2024-09-21T11:44:20.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:44:20.541 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:44:20.559 INFO:teuthology.orchestra.run.smithi193.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 291 kB/s | 107 kB 00:00 2024-09-21T11:44:20.585 INFO:teuthology.orchestra.run.smithi193.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 465 kB/s | 171 kB 00:00 2024-09-21T11:44:20.605 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:44:20.610 INFO:teuthology.orchestra.run.smithi193.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-21T11:44:20.635 INFO:teuthology.orchestra.run.smithi193.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 767 kB/s | 160 kB 00:00 2024-09-21T11:44:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-21T11:44:20.685 INFO:teuthology.orchestra.run.smithi193.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 843 kB/s | 42 kB 00:00 2024-09-21T11:44:20.711 INFO:teuthology.orchestra.run.smithi193.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-21T11:44:20.736 INFO:teuthology.orchestra.run.smithi193.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T11:44:20.761 INFO:teuthology.orchestra.run.smithi193.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-21T11:44:20.767 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-21T11:44:20.787 INFO:teuthology.orchestra.run.smithi193.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-21T11:44:20.804 INFO:teuthology.orchestra.run.smithi193.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 703 kB/s | 47 kB 00:00 2024-09-21T11:44:20.824 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-21T11:44:20.854 INFO:teuthology.orchestra.run.smithi193.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.3 MB/s | 308 kB 00:00 2024-09-21T11:44:20.866 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-21T11:44:20.871 INFO:teuthology.orchestra.run.smithi193.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-21T11:44:20.888 INFO:teuthology.orchestra.run.smithi193.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 374 kB/s | 26 kB 00:00 2024-09-21T11:44:20.913 INFO:teuthology.orchestra.run.smithi193.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-21T11:44:20.915 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-21T11:44:20.944 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-21T11:44:20.963 INFO:teuthology.orchestra.run.smithi193.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-21T11:44:20.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:44:21.005 INFO:teuthology.orchestra.run.smithi193.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-21T11:44:21.029 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-21T11:44:21.064 INFO:teuthology.orchestra.run.smithi193.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T11:44:21.139 INFO:teuthology.orchestra.run.smithi193.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-21T11:44:21.289 INFO:teuthology.orchestra.run.smithi193.stdout:(32/33): librados2-19.3.0-5103.g5c1cdc3c.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-21T11:44:21.348 INFO:teuthology.orchestra.run.smithi193.stdout:(33/33): librbd1-19.3.0-5103.g5c1cdc3c.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-21T11:44:21.353 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:44:21.353 INFO:teuthology.orchestra.run.smithi193.stdout:Total 23 MB/s | 62 MB 00:02 2024-09-21T11:44:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-21T11:44:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:44:21.467 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:44:21.496 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:44:21.503 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:44:21.503 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:44:21.537 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 24/35 2024-09-21T11:44:21.582 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-21T11:44:22.007 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:44:22.007 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:44:22.072 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-21T11:44:22.110 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:44:22.196 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:44:22.288 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 28/35 2024-09-21T11:44:22.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 29/35 2024-09-21T11:44:23.638 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:44:23.724 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-21T11:44:23.770 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-21T11:44:23.803 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-21T11:44:23.826 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:44:23.849 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:44:24.002 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:44:24.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:44:24.439 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-21T11:44:24.502 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:44:24.594 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:44:24.681 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:44:24.691 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 8/35 2024-09-21T11:44:24.718 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:44:24.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:44:24.807 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:44:24.836 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-21T11:44:24.879 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 11/35 2024-09-21T11:44:24.948 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:44:24.972 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:44:25.019 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:44:25.054 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-21T11:44:25.163 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-21T11:44:25.197 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-21T11:44:25.244 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-21T11:44:25.278 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-21T11:44:25.299 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:44:25.307 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-21T11:44:25.338 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:44:25.341 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:44:25.386 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-21T11:44:25.429 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:44:25.429 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T11:44:25.429 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T11:44:25.429 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:25.464 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:44:25.704 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-21T11:44:25.746 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:44:25.829 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:44:25.869 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 24/35 2024-09-21T11:44:25.912 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-21T11:44:26.408 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-21T11:44:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:44:26.519 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:44:26.602 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 28/35 2024-09-21T11:44:26.634 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 29/35 2024-09-21T11:44:28.975 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:44:29.014 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:44:29.587 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:44:29.626 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:44:29.716 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:44:29.717 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T11:44:29.717 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T11:44:29.717 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:29.752 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:44:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:44:35.685 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-21T11:44:35.685 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-21T11:44:35.685 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-21T11:44:35.685 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-21T11:44:35.686 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-21T11:44:35.686 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-21T11:44:35.686 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-21T11:44:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:36.708 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:44:36.734 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:44:36.735 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:44:36.735 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T11:44:36.735 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:44:36.735 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:44:36.735 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:36.736 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:44:36.827 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:44:36.827 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 6/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 8/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 9/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 10/35 2024-09-21T11:44:38.013 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 11/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 13/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-21T11:44:38.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-21T11:44:38.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/35 2024-09-21T11:44:38.906 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:44:38.906 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:38.906 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-21T11:44:38.906 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.906 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.906 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T11:44:38.907 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T11:44:38.908 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T11:44:38.909 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:38.909 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:44:39.298 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-09-21T11:44:39.871 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:39.883 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:27 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:44:40.021 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:44:40.021 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:40.021 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:44:40.021 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 46 M 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-21T11:44:40.022 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:40.023 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 46 M 2024-09-21T11:44:40.023 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 201 M 2024-09-21T11:44:40.023 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:44:40.335 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-21T11:44:40.360 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): libcephsqlite-19.3.0-5103.g5c1cdc3c.el9. 694 kB/s | 161 kB 00:00 2024-09-21T11:44:40.385 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T11:44:40.852 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:44:40.877 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:44:40.877 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:44:40.877 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T11:44:40.877 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:44:40.877 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:44:40.877 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:40.879 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:44:40.968 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:44:40.968 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:44:41.661 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): ceph-test-19.3.0-5103.g5c1cdc3c.el9.x86_ 30 MB/s | 46 MB 00:01 2024-09-21T11:44:41.663 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:44:41.663 INFO:teuthology.orchestra.run.smithi107.stdout:Total 28 MB/s | 46 MB 00:01 2024-09-21T11:44:41.690 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:44:41.710 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:44:41.710 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:44:42.075 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:44:42.075 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:44:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:44:42.085 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 6/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 8/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 9/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 10/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 11/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 13/35 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-21T11:44:42.087 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-21T11:44:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-21T11:44:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-21T11:44:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-21T11:44:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:44:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-21T11:44:42.088 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/35 2024-09-21T11:44:42.636 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:44:42.735 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T11:44:42.785 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T11:44:42.813 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:44:43.031 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:44:43.032 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:43.032 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-09-21T11:44:43.032 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.032 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.032 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:44:43.032 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T11:44:43.033 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.033 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.033 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.033 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.033 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T11:44:43.033 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T11:44:43.034 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.035 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T11:44:43.035 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.035 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T11:44:43.035 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T11:44:43.035 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T11:44:43.035 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.036 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.036 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.036 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T11:44:43.036 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.036 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.036 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:43.037 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:44:43.442 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-test 2024-09-21T11:44:44.037 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:28 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:44:44.168 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 46 M 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:44.169 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:44:44.170 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:44.170 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-09-21T11:44:44.170 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:44.170 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 46 M 2024-09-21T11:44:44.170 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 201 M 2024-09-21T11:44:44.170 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:44:44.504 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-21T11:44:44.529 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): libcephsqlite-19.3.0-5103.g5c1cdc3c.el9. 680 kB/s | 161 kB 00:00 2024-09-21T11:44:44.554 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T11:44:45.747 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): ceph-test-19.3.0-5103.g5c1cdc3c.el9.x86_ 31 MB/s | 46 MB 00:01 2024-09-21T11:44:45.749 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:44:45.749 INFO:teuthology.orchestra.run.smithi193.stdout:Total 29 MB/s | 46 MB 00:01 2024-09-21T11:44:45.777 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:44:45.797 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:44:45.798 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:44:46.166 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:44:46.167 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:44:46.727 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:44:46.808 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T11:44:46.855 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T11:44:46.882 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:44:47.158 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:44:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:44:49.507 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:44:49.507 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/4 2024-09-21T11:44:49.507 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/4 2024-09-21T11:44:49.507 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T11:44:49.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:49.804 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:44:50.076 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-09-21T11:44:50.659 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:38 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:44:50.799 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 6.6 k 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 2.4 M 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 1.5 M 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 292 k 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 4.8 M 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 17 M 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T11:44:50.802 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T11:44:50.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T11:44:50.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout:Install 37 Packages 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 31 M 2024-09-21T11:44:50.805 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 109 M 2024-09-21T11:44:50.806 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:44:51.223 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:44:51.297 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:44:51.709 INFO:teuthology.orchestra.run.smithi107.stdout:(1/37): ceph-19.3.0-5103.g5c1cdc3c.el9.x86_64.r 27 kB/s | 6.6 kB 00:00 2024-09-21T11:44:51.901 INFO:teuthology.orchestra.run.smithi107.stdout:(2/37): ceph-mds-19.3.0-5103.g5c1cdc3c.el9.x86_ 5.5 MB/s | 2.4 MB 00:00 2024-09-21T11:44:51.935 INFO:teuthology.orchestra.run.smithi107.stdout:(3/37): ceph-mgr-19.3.0-5103.g5c1cdc3c.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-09-21T11:44:52.043 INFO:teuthology.orchestra.run.smithi107.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5103.g5c1c 2.6 MB/s | 292 kB 00:00 2024-09-21T11:44:52.161 INFO:teuthology.orchestra.run.smithi107.stdout:(5/37): ceph-mon-19.3.0-5103.g5c1cdc3c.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-21T11:44:52.211 INFO:teuthology.orchestra.run.smithi107.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-21T11:44:52.278 INFO:teuthology.orchestra.run.smithi107.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T11:44:52.320 INFO:teuthology.orchestra.run.smithi107.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 7.7 MB/s | 1.2 MB 00:00 2024-09-21T11:44:52.495 INFO:teuthology.orchestra.run.smithi107.stdout:(9/37): ceph-osd-19.3.0-5103.g5c1cdc3c.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-21T11:44:52.529 INFO:teuthology.orchestra.run.smithi107.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 598 kB/s | 125 kB 00:00 2024-09-21T11:44:52.554 INFO:teuthology.orchestra.run.smithi107.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 490 kB/s | 135 kB 00:00 2024-09-21T11:44:52.654 INFO:teuthology.orchestra.run.smithi107.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-09-21T11:44:52.680 INFO:teuthology.orchestra.run.smithi107.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-21T11:44:52.705 INFO:teuthology.orchestra.run.smithi107.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-21T11:44:52.730 INFO:teuthology.orchestra.run.smithi107.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-21T11:44:52.755 INFO:teuthology.orchestra.run.smithi107.stdout:(16/37): python3-markupsafe-1.1.1-12.el9.x86_64 344 kB/s | 35 kB 00:00 2024-09-21T11:44:52.781 INFO:teuthology.orchestra.run.smithi107.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-21T11:44:52.806 INFO:teuthology.orchestra.run.smithi107.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-21T11:44:52.848 INFO:teuthology.orchestra.run.smithi107.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-21T11:44:52.882 INFO:teuthology.orchestra.run.smithi107.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-21T11:44:52.907 INFO:teuthology.orchestra.run.smithi107.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-21T11:44:52.932 INFO:teuthology.orchestra.run.smithi107.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-21T11:44:52.957 INFO:teuthology.orchestra.run.smithi107.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 703 kB/s | 18 kB 00:00 2024-09-21T11:44:52.983 INFO:teuthology.orchestra.run.smithi107.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-21T11:44:53.008 INFO:teuthology.orchestra.run.smithi107.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-21T11:44:53.025 INFO:teuthology.orchestra.run.smithi107.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 458 kB/s | 19 kB 00:00 2024-09-21T11:44:53.042 INFO:teuthology.orchestra.run.smithi107.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 582 kB/s | 20 kB 00:00 2024-09-21T11:44:53.059 INFO:teuthology.orchestra.run.smithi107.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-21T11:44:53.085 INFO:teuthology.orchestra.run.smithi107.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-21T11:44:53.127 INFO:teuthology.orchestra.run.smithi107.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 937 kB/s | 79 kB 00:00 2024-09-21T11:44:53.152 INFO:teuthology.orchestra.run.smithi107.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 246 kB/s | 16 kB 00:00 2024-09-21T11:44:53.178 INFO:teuthology.orchestra.run.smithi107.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-21T11:44:53.203 INFO:teuthology.orchestra.run.smithi107.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-21T11:44:53.228 INFO:teuthology.orchestra.run.smithi107.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-21T11:44:53.253 INFO:teuthology.orchestra.run.smithi107.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T11:44:53.278 INFO:teuthology.orchestra.run.smithi107.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-21T11:44:53.304 INFO:teuthology.orchestra.run.smithi107.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-21T11:44:53.308 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:44:53.308 INFO:teuthology.orchestra.run.smithi107.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-21T11:44:53.457 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:44:53.498 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:44:53.498 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:44:53.713 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:44:53.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/4 2024-09-21T11:44:53.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/4 2024-09-21T11:44:53.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T11:44:53.952 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:44:53.952 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T11:44:53.982 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T11:44:53.983 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:53.983 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:44:54.259 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph 2024-09-21T11:44:54.840 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:38 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:44:54.972 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:44:54.974 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:54.974 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:44:54.974 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:54.974 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 6.6 k 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 2.4 M 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 1.5 M 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 292 k 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 4.8 M 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 17 M 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T11:44:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T11:44:54.976 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T11:44:54.977 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout:Install 37 Packages 2024-09-21T11:44:54.978 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:44:54.979 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 31 M 2024-09-21T11:44:54.979 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 109 M 2024-09-21T11:44:54.979 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:44:55.254 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:44:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T11:44:55.356 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T11:44:55.383 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T11:44:55.457 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T11:44:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T11:44:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T11:44:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T11:44:55.745 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T11:44:55.785 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T11:44:55.820 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T11:44:55.850 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T11:44:55.893 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T11:44:55.926 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T11:44:55.967 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T11:44:56.018 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T11:44:56.028 INFO:teuthology.orchestra.run.smithi193.stdout:(1/37): ceph-19.3.0-5103.g5c1cdc3c.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-21T11:44:56.061 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T11:44:56.110 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T11:44:56.149 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T11:44:56.224 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T11:44:56.228 INFO:teuthology.orchestra.run.smithi193.stdout:(2/37): ceph-mgr-19.3.0-5103.g5c1cdc3c.el9.x86_ 4.7 MB/s | 1.5 MB 00:00 2024-09-21T11:44:56.279 INFO:teuthology.orchestra.run.smithi193.stdout:(3/37): ceph-mds-19.3.0-5103.g5c1cdc3c.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-21T11:44:56.362 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T11:44:56.379 INFO:teuthology.orchestra.run.smithi193.stdout:(4/37): ceph-mon-19.3.0-5103.g5c1cdc3c.el9.x86_ 14 MB/s | 4.8 MB 00:00 2024-09-21T11:44:56.404 INFO:teuthology.orchestra.run.smithi193.stdout:(5/37): ceph-mgr-modules-core-19.3.0-5103.g5c1c 2.3 MB/s | 292 kB 00:00 2024-09-21T11:44:56.450 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T11:44:56.512 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T11:44:56.550 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T11:44:56.600 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T11:44:56.654 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T11:44:56.805 INFO:teuthology.orchestra.run.smithi193.stdout:(6/37): ceph-osd-19.3.0-5103.g5c1cdc3c.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-21T11:44:56.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T11:44:56.913 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T11:44:56.991 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T11:44:57.081 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T11:44:57.126 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T11:44:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T11:44:57.330 INFO:teuthology.orchestra.run.smithi193.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 266 kB/s | 253 kB 00:00 2024-09-21T11:44:57.426 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 32/37 2024-09-21T11:44:57.456 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:44:57.463 INFO:teuthology.orchestra.run.smithi193.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 1.2 MB/s | 1.2 MB 00:01 2024-09-21T11:44:57.483 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:44:57.483 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:44:57.483 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T11:44:57.483 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:44:57.483 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:44:57.483 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:57.580 INFO:teuthology.orchestra.run.smithi193.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 138 kB/s | 106 kB 00:00 2024-09-21T11:44:58.147 INFO:teuthology.orchestra.run.smithi193.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 165 kB/s | 135 kB 00:00 2024-09-21T11:44:58.206 INFO:teuthology.orchestra.run.smithi193.stdout:(11/37): python3-requests-2.25.1-8.el9.noarch.r 168 kB/s | 125 kB 00:00 2024-09-21T11:44:58.398 INFO:teuthology.orchestra.run.smithi193.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 263 kB/s | 215 kB 00:00 2024-09-21T11:44:58.673 INFO:teuthology.orchestra.run.smithi193.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 126 kB/s | 35 kB 00:00 2024-09-21T11:44:58.698 INFO:teuthology.orchestra.run.smithi193.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 342 kB/s | 188 kB 00:00 2024-09-21T11:44:58.790 INFO:teuthology.orchestra.run.smithi193.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 295 kB/s | 172 kB 00:00 2024-09-21T11:44:58.807 INFO:teuthology.orchestra.run.smithi193.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-21T11:44:58.874 INFO:teuthology.orchestra.run.smithi193.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 384 kB/s | 77 kB 00:00 2024-09-21T11:44:58.891 INFO:teuthology.orchestra.run.smithi193.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-21T11:44:58.940 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:44:58.941 INFO:teuthology.orchestra.run.smithi193.stdout:(19/37): python3-toml-0.10.2-6.el9.noarch.rpm 275 kB/s | 42 kB 00:00 2024-09-21T11:44:58.967 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:44:58.967 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:44:58.967 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T11:44:58.967 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:44:58.967 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:44:58.967 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:58.992 INFO:teuthology.orchestra.run.smithi193.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-09-21T11:44:59.017 INFO:teuthology.orchestra.run.smithi193.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 346 kB/s | 43 kB 00:00 2024-09-21T11:44:59.042 INFO:teuthology.orchestra.run.smithi193.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 463 kB/s | 11 kB 00:00 2024-09-21T11:44:59.067 INFO:teuthology.orchestra.run.smithi193.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 713 kB/s | 18 kB 00:00 2024-09-21T11:44:59.093 INFO:teuthology.orchestra.run.smithi193.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 934 kB/s | 23 kB 00:00 2024-09-21T11:44:59.118 INFO:teuthology.orchestra.run.smithi193.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-21T11:44:59.143 INFO:teuthology.orchestra.run.smithi193.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 385 kB/s | 19 kB 00:00 2024-09-21T11:44:59.168 INFO:teuthology.orchestra.run.smithi193.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-21T11:44:59.194 INFO:teuthology.orchestra.run.smithi193.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-21T11:44:59.219 INFO:teuthology.orchestra.run.smithi193.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-21T11:44:59.245 INFO:teuthology.orchestra.run.smithi193.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-21T11:44:59.270 INFO:teuthology.orchestra.run.smithi193.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-21T11:44:59.295 INFO:teuthology.orchestra.run.smithi193.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-21T11:44:59.320 INFO:teuthology.orchestra.run.smithi193.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-21T11:44:59.346 INFO:teuthology.orchestra.run.smithi193.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-21T11:44:59.371 INFO:teuthology.orchestra.run.smithi193.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-09-21T11:44:59.396 INFO:teuthology.orchestra.run.smithi193.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T11:44:59.416 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:44:59.430 INFO:teuthology.orchestra.run.smithi193.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-21T11:44:59.435 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:44:59.435 INFO:teuthology.orchestra.run.smithi193.stdout:Total 6.9 MB/s | 31 MB 00:04 2024-09-21T11:44:59.439 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:44:59.439 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:44:59.440 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T11:44:59.440 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:44:59.440 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:44:59.440 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:59.585 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:44:59.625 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:44:59.625 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:44:59.667 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:44:59.691 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:44:59.691 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:44:59.691 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T11:44:59.691 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:44:59.691 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:44:59.691 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:44:59.756 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:45:00.089 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:00.089 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:01.383 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:01.449 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T11:45:01.482 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T11:45:01.510 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T11:45:01.523 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 6/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T11:45:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T11:45:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T11:45:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T11:45:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T11:45:01.527 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T11:45:01.580 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T11:45:01.651 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T11:45:01.683 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T11:45:01.774 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T11:45:01.853 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T11:45:01.894 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T11:45:01.929 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T11:45:01.959 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T11:45:02.003 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T11:45:02.035 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T11:45:02.076 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T11:45:02.111 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T11:45:02.146 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T11:45:02.196 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T11:45:02.232 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T11:45:02.309 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T11:45:02.446 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T11:45:02.527 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T11:45:02.529 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T11:45:02.529 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:02.529 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T11:45:02.530 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T11:45:02.531 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T11:45:02.532 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T11:45:02.533 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T11:45:02.533 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T11:45:02.533 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T11:45:02.533 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T11:45:02.533 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:02.533 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:02.581 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T11:45:02.753 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T11:45:02.802 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T11:45:02.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-09-21T11:45:02.857 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T11:45:03.058 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T11:45:03.108 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T11:45:03.185 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T11:45:03.334 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T11:45:03.426 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:51 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:03.459 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T11:45:03.515 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:03.556 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:03.557 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:03.557 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:03.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-09-21T11:45:03.629 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 32/37 2024-09-21T11:45:03.659 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:45:03.684 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:45:03.685 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:03.685 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T11:45:03.685 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:45:03.685 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:45:03.685 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:04.212 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:52 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:04.342 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 784 k 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:04.344 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 784 k 2024-09-21T11:45:04.344 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 791 k 2024-09-21T11:45:04.344 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:04.635 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-19.3.0-5103.g5c1cdc3c.el9.noarch.rpm 2.6 MB/s | 784 kB 00:00 2024-09-21T11:45:04.636 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:04.636 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.6 MB/s | 784 kB 00:00 2024-09-21T11:45:04.636 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:04.641 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:04.641 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:04.647 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:04.647 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:04.717 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:05.144 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:45:05.176 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:45:05.176 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:05.176 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T11:45:05.176 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:45:05.176 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:45:05.176 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:05.180 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:05.255 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:05.636 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:05.645 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:45:05.671 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:45:05.671 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:05.671 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T11:45:05.671 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:45:05.671 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:45:05.671 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:05.876 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:05.876 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:05.876 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:05.876 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:05.877 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:05.905 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:45:05.929 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:45:05.930 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:05.930 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T11:45:05.930 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:45:05.930 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:45:05.930 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:05.995 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:45:06.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T11:45:06.716 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:06.850 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:06.850 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-21T11:45:06.850 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:06.850 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 145 k 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 145 k 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 432 k 2024-09-21T11:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:07.058 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-19.3.0-5103.g5c1cdc 703 kB/s | 145 kB 00:00 2024-09-21T11:45:07.058 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:07.059 INFO:teuthology.orchestra.run.smithi107.stdout:Total 697 kB/s | 145 kB 00:00 2024-09-21T11:45:07.059 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:07.066 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:07.066 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:07.123 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:07.123 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:07.268 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:07.326 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:45:07.351 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:45:07.351 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:07.351 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T11:45:07.351 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:45:07.710 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 6/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T11:45:07.711 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T11:45:07.712 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T11:45:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T11:45:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T11:45:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T11:45:07.713 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T11:45:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T11:45:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T11:45:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T11:45:07.714 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T11:45:08.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:45:08.284 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:08.284 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:08.284 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:08.284 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:08.284 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:08.512 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-09-21T11:45:08.820 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T11:45:08.820 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:08.820 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:08.820 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:08.820 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T11:45:08.821 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T11:45:08.822 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:08.823 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:09.117 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:09.207 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:09.249 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:09.250 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:09.250 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:09.268 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-base 2024-09-21T11:45:09.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T11:45:09.866 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:09.902 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:09.956 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:09.999 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:10.000 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:10.000 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:10.035 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:10.036 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:10.036 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 14 M 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 34 k 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 16 k 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T11:45:10.037 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-09-21T11:45:10.038 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:10.039 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 19 M 2024-09-21T11:45:10.039 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 159 M 2024-09-21T11:45:10.039 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:10.084 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install cephadm 2024-09-21T11:45:10.421 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): ceph-grafana-dashboards-19.3.0-5103.g5c 220 kB/s | 34 kB 00:00 2024-09-21T11:45:10.446 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): ceph-prometheus-alerts-19.3.0-5103.g5c1 91 kB/s | 16 kB 00:00 2024-09-21T11:45:10.613 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-09-21T11:45:10.660 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:10.671 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-21T11:45:10.722 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-21T11:45:10.738 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-21T11:45:10.789 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-09-21T11:45:10.791 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 784 k 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:45:10.793 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 784 k 2024-09-21T11:45:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 791 k 2024-09-21T11:45:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:10.831 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-09-21T11:45:10.856 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-21T11:45:10.898 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 13 MB/s | 2.0 MB 00:00 2024-09-21T11:45:10.923 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-21T11:45:11.057 INFO:teuthology.orchestra.run.smithi193.stdout:cephadm-19.3.0-5103.g5c1cdc3c.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-21T11:45:11.058 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:11.058 INFO:teuthology.orchestra.run.smithi193.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-21T11:45:11.058 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:11.063 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:11.063 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:11.069 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:11.070 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:11.073 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): ceph-mgr-dashboard-19.3.0-5103.g5c1cdc 18 MB/s | 14 MB 00:00 2024-09-21T11:45:11.078 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:11.078 INFO:teuthology.orchestra.run.smithi107.stdout:Total 19 MB/s | 19 MB 00:01 2024-09-21T11:45:11.126 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:11.139 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:11.145 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:11.145 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:11.414 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:11.414 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:11.578 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:11.687 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:12.062 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:12.100 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T11:45:12.135 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T11:45:12.181 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T11:45:12.212 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T11:45:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T11:45:12.313 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:12.313 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:12.313 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:12.313 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:12.313 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:12.313 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:12.440 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T11:45:12.531 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T11:45:12.532 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T11:45:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T11:45:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:45:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 10/12 2024-09-21T11:45:13.125 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:13.260 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:13.260 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:13.260 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-09-21T11:45:13.260 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:13.260 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 145 k 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:13.261 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 145 k 2024-09-21T11:45:13.262 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 432 k 2024-09-21T11:45:13.262 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:13.465 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-immutable-object-cache-19.3.0-5103.g5c1cdc 715 kB/s | 145 kB 00:00 2024-09-21T11:45:13.465 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:13.465 INFO:teuthology.orchestra.run.smithi193.stdout:Total 709 kB/s | 145 kB 00:00 2024-09-21T11:45:13.466 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:13.473 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:13.473 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:13.531 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:13.531 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:13.691 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:13.774 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:45:13.800 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:45:13.800 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:13.800 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T11:45:13.800 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:14.719 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:45:14.719 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:14.719 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:14.719 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:14.719 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:14.719 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:14.936 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr 2024-09-21T11:45:15.528 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:15.621 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:15.663 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:15.664 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:15.664 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:15.729 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T11:45:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 11/12 2024-09-21T11:45:15.941 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:45:16.312 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:00 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:16.449 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:16.450 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:16.450 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:16.450 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:16.450 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:16.450 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 14 M 2024-09-21T11:45:16.450 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 34 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 16 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T11:45:16.451 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T11:45:16.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T11:45:16.452 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:16.452 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:16.452 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:16.452 INFO:teuthology.orchestra.run.smithi193.stdout:Install 12 Packages 2024-09-21T11:45:16.452 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:16.453 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 19 M 2024-09-21T11:45:16.453 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 159 M 2024-09-21T11:45:16.453 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:16.944 INFO:teuthology.orchestra.run.smithi193.stdout:(1/12): ceph-grafana-dashboards-19.3.0-5103.g5c 220 kB/s | 34 kB 00:00 2024-09-21T11:45:16.969 INFO:teuthology.orchestra.run.smithi193.stdout:(2/12): ceph-prometheus-alerts-19.3.0-5103.g5c1 91 kB/s | 16 kB 00:00 2024-09-21T11:45:17.170 INFO:teuthology.orchestra.run.smithi193.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-21T11:45:17.295 INFO:teuthology.orchestra.run.smithi193.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-21T11:45:17.329 INFO:teuthology.orchestra.run.smithi193.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.3 MB/s | 863 kB 00:00 2024-09-21T11:45:17.371 INFO:teuthology.orchestra.run.smithi193.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 7.1 MB/s | 548 kB 00:00 2024-09-21T11:45:17.404 INFO:teuthology.orchestra.run.smithi193.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 264 kB/s | 19 kB 00:00 2024-09-21T11:45:17.563 INFO:teuthology.orchestra.run.smithi193.stdout:(8/12): ceph-mgr-dashboard-19.3.0-5103.g5c1cdc3 18 MB/s | 14 MB 00:00 2024-09-21T11:45:17.588 INFO:teuthology.orchestra.run.smithi193.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 787 kB/s | 144 kB 00:00 2024-09-21T11:45:17.630 INFO:teuthology.orchestra.run.smithi193.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.3 MB/s | 188 kB 00:00 2024-09-21T11:45:17.655 INFO:teuthology.orchestra.run.smithi193.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 335 kB/s | 31 kB 00:00 2024-09-21T11:45:17.697 INFO:teuthology.orchestra.run.smithi193.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.3 MB/s | 2.0 MB 00:00 2024-09-21T11:45:17.701 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:17.701 INFO:teuthology.orchestra.run.smithi193.stdout:Total 15 MB/s | 19 MB 00:01 2024-09-21T11:45:17.751 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:17.770 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:17.770 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 1/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 2/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 3/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:45:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T11:45:17.963 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T11:45:18.034 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:18.035 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:18.278 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T11:45:18.279 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T11:45:18.280 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T11:45:18.280 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:18.280 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:18.530 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T11:45:18.627 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:18.698 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T11:45:18.741 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T11:45:18.788 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T11:45:18.835 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T11:45:18.930 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T11:45:19.061 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T11:45:19.106 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:07 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:19.164 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T11:45:19.207 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T11:45:19.235 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:45:19.238 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:19.239 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-21T11:45:19.239 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:19.239 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-21T11:45:19.239 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:19.239 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 7.4 M 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 38 M 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 212 M 2024-09-21T11:45:19.241 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 10/12 2024-09-21T11:45:20.023 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-51 16 MB/s | 7.4 MB 00:00 2024-09-21T11:45:20.390 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 232 kB/s | 191 kB 00:00 2024-09-21T11:45:20.465 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 892 kB/s | 801 kB 00:00 2024-09-21T11:45:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-21T11:45:20.932 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 39 kB/s | 18 kB 00:00 2024-09-21T11:45:21.133 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-21T11:45:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-21T11:45:21.917 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 312 kB/s | 245 kB 00:00 2024-09-21T11:45:22.084 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.3 MB/s | 4.9 MB 00:01 2024-09-21T11:45:22.459 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 11/12 2024-09-21T11:45:22.506 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:45:22.584 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-21T11:45:22.717 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 313 kB/s | 250 kB 00:00 2024-09-21T11:45:23.559 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T11:45:23.561 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:23.561 INFO:teuthology.orchestra.run.smithi107.stdout:Total 8.8 MB/s | 38 MB 00:04 2024-09-21T11:45:23.921 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:23.947 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:23.948 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:24.402 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:24.402 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 1/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 2/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 3/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T11:45:24.513 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:45:24.514 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T11:45:24.514 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T11:45:24.850 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T11:45:24.850 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:24.850 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:24.850 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T11:45:24.850 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:24.850 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:24.851 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:25.114 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T11:45:25.160 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:25.231 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T11:45:25.274 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:45:25.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:45:25.408 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T11:45:25.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T11:45:25.624 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T11:45:25.689 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:09 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T11:45:25.821 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 7.4 M 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T11:45:25.822 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout:Install 12 Packages 2024-09-21T11:45:25.823 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:25.824 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 38 M 2024-09-21T11:45:25.824 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 212 M 2024-09-21T11:45:25.824 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:26.255 INFO:teuthology.orchestra.run.smithi193.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 930 kB/s | 191 kB 00:00 2024-09-21T11:45:26.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T11:45:26.389 INFO:teuthology.orchestra.run.smithi193.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-21T11:45:26.414 INFO:teuthology.orchestra.run.smithi193.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-09-21T11:45:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T11:45:26.548 INFO:teuthology.orchestra.run.smithi193.stdout:(4/12): ceph-mgr-diskprediction-local-19.3.0-51 15 MB/s | 7.4 MB 00:00 2024-09-21T11:45:26.573 INFO:teuthology.orchestra.run.smithi193.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 115 kB/s | 18 kB 00:00 2024-09-21T11:45:26.665 INFO:teuthology.orchestra.run.smithi193.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-21T11:45:26.807 INFO:teuthology.orchestra.run.smithi193.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2024-09-21T11:45:26.849 INFO:teuthology.orchestra.run.smithi193.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-21T11:45:26.932 INFO:teuthology.orchestra.run.smithi193.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-21T11:45:26.957 INFO:teuthology.orchestra.run.smithi193.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-09-21T11:45:27.033 INFO:teuthology.orchestra.run.smithi193.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-21T11:45:27.350 INFO:teuthology.orchestra.run.smithi193.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-21T11:45:27.351 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:27.351 INFO:teuthology.orchestra.run.smithi193.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-21T11:45:27.704 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:45:27.729 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:27.729 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:28.183 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:28.183 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:28.736 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T11:45:28.767 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:45:28.963 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:29.028 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T11:45:29.064 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:45:29.161 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:45:29.199 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T11:45:29.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T11:45:29.423 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T11:45:29.499 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T11:45:30.147 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T11:45:30.260 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T11:45:31.477 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:45:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 1/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:45:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T11:45:32.097 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T11:45:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T11:45:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T11:45:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T11:45:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T11:45:32.098 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:32.098 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:32.499 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-09-21T11:45:32.500 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T11:45:32.524 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:45:33.089 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:21 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:33.224 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 49 k 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:45:33.225 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:33.226 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:33.227 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:33.227 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-09-21T11:45:33.227 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:33.227 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.2 M 2024-09-21T11:45:33.227 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-09-21T11:45:33.227 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:33.486 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 343 kB/s | 26 kB 00:00 2024-09-21T11:45:33.511 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 191 kB/s | 19 kB 00:00 2024-09-21T11:45:33.603 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): ceph-mgr-rook-19.3.0-5103.g5c1cdc3c.el9 257 kB/s | 49 kB 00:00 2024-09-21T11:45:33.628 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T11:45:33.654 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-21T11:45:33.679 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-21T11:45:33.721 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-21T11:45:33.746 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.9 MB/s | 279 kB 00:00 2024-09-21T11:45:33.772 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-21T11:45:33.805 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-21T11:45:33.831 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-21T11:45:33.864 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-21T11:45:33.906 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-21T11:45:33.909 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:33.909 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-21T11:45:33.971 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:34.073 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:34.073 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:34.805 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T11:45:34.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T11:45:34.921 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T11:45:34.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 1/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T11:45:34.993 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T11:45:34.994 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T11:45:34.994 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T11:45:34.994 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:45:34.994 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T11:45:35.000 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T11:45:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T11:45:35.238 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T11:45:35.294 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T11:45:35.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T11:45:35.461 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:35.461 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:35.461 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:35.461 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T11:45:35.461 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:35.462 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:35.859 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-rook 2024-09-21T11:45:36.003 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T11:45:36.058 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T11:45:36.091 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T11:45:36.140 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T11:45:36.170 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:45:36.465 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:20 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:36.605 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:36.606 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 49 k 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T11:45:36.607 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout:Install 13 Packages 2024-09-21T11:45:36.608 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:36.609 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 2.2 M 2024-09-21T11:45:36.609 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 26 M 2024-09-21T11:45:36.609 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:36.926 INFO:teuthology.orchestra.run.smithi193.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 211 kB/s | 19 kB 00:00 2024-09-21T11:45:36.951 INFO:teuthology.orchestra.run.smithi193.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 221 kB/s | 26 kB 00:00 2024-09-21T11:45:36.990 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:45:36.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/13 2024-09-21T11:45:36.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T11:45:36.990 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T11:45:36.991 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T11:45:37.009 INFO:teuthology.orchestra.run.smithi193.stdout:(3/13): ceph-mgr-rook-19.3.0-5103.g5c1cdc3c.el9 282 kB/s | 49 kB 00:00 2024-09-21T11:45:37.060 INFO:teuthology.orchestra.run.smithi193.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-21T11:45:37.085 INFO:teuthology.orchestra.run.smithi193.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-21T11:45:37.110 INFO:teuthology.orchestra.run.smithi193.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-21T11:45:37.152 INFO:teuthology.orchestra.run.smithi193.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-21T11:45:37.177 INFO:teuthology.orchestra.run.smithi193.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-21T11:45:37.203 INFO:teuthology.orchestra.run.smithi193.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 153 kB/s | 14 kB 00:00 2024-09-21T11:45:37.236 INFO:teuthology.orchestra.run.smithi193.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-21T11:45:37.262 INFO:teuthology.orchestra.run.smithi193.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-21T11:45:37.278 INFO:teuthology.orchestra.run.smithi193.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.2 MB/s | 90 kB 00:00 2024-09-21T11:45:37.320 INFO:teuthology.orchestra.run.smithi193.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-21T11:45:37.323 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:37.323 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-21T11:45:37.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T11:45:37.327 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:37.327 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:37.327 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:37.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:37.328 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:37.390 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:37.405 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:37.405 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:37.497 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:37.497 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:37.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T11:45:38.019 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:38.170 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:26 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:38.192 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T11:45:38.246 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T11:45:38.301 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 154 k 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:45:38.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-09-21T11:45:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:38.304 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T11:45:38.304 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 7.0 M 2024-09-21T11:45:38.304 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-09-21T11:45:38.304 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:38.340 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T11:45:38.381 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T11:45:38.482 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T11:45:38.592 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T11:45:38.641 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T11:45:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): ceph-mgr-cephadm-19.3.0-5103.g5c1cdc3c.e 724 kB/s | 154 kB 00:00 2024-09-21T11:45:39.001 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-21T11:45:39.018 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.3 MB/s | 58 kB 00:00 2024-09-21T11:45:39.043 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-21T11:45:39.356 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T11:45:39.406 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T11:45:39.439 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T11:45:39.487 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T11:45:39.510 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:45:39.560 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 299 kB/s | 248 kB 00:00 2024-09-21T11:45:39.977 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-21T11:45:39.979 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:39.979 INFO:teuthology.orchestra.run.smithi107.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-21T11:45:40.050 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:40.061 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:40.061 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:40.160 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:40.160 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T11:45:40.258 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T11:45:40.259 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T11:45:40.259 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T11:45:40.259 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T11:45:40.259 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T11:45:40.259 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T11:45:40.487 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T11:45:40.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T11:45:40.591 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T11:45:40.591 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T11:45:40.591 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T11:45:40.591 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T11:45:40.591 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:40.591 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T11:45:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T11:45:40.861 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T11:45:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T11:45:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T11:45:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T11:45:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:45:41.442 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:41.574 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 154 k 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T11:45:41.575 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout:Install 6 Packages 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 7.0 M 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 33 M 2024-09-21T11:45:41.576 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:41.932 INFO:teuthology.orchestra.run.smithi193.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-21T11:45:41.982 INFO:teuthology.orchestra.run.smithi193.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-21T11:45:42.016 INFO:teuthology.orchestra.run.smithi193.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-09-21T11:45:42.041 INFO:teuthology.orchestra.run.smithi193.stdout:(4/6): ceph-mgr-cephadm-19.3.0-5103.g5c1cdc3c.e 647 kB/s | 154 kB 00:00 2024-09-21T11:45:42.067 INFO:teuthology.orchestra.run.smithi193.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-21T11:45:42.159 INFO:teuthology.orchestra.run.smithi193.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-21T11:45:42.160 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:42.161 INFO:teuthology.orchestra.run.smithi193.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-21T11:45:42.231 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:42.242 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:42.242 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/6 2024-09-21T11:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T11:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T11:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T11:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T11:45:42.343 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:42.343 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:42.526 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T11:45:42.526 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:42.526 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:42.526 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:42.527 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:42.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-09-21T11:45:42.828 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T11:45:42.886 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T11:45:43.378 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:43.423 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T11:45:43.511 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 899 k 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:43.512 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:43.513 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-09-21T11:45:43.513 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:43.513 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 979 k 2024-09-21T11:45:43.513 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.9 M 2024-09-21T11:45:43.513 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:43.515 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T11:45:43.590 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T11:45:43.615 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:45:43.947 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 437 kB/s | 80 kB 00:00 2024-09-21T11:45:44.039 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): ceph-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_ 3.2 MB/s | 899 kB 00:00 2024-09-21T11:45:44.039 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:44.040 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.8 MB/s | 979 kB 00:00 2024-09-21T11:45:44.051 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:44.074 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:44.074 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:44.157 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:44.158 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:44.354 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:45:44.354 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/6 2024-09-21T11:45:44.354 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T11:45:44.354 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T11:45:44.354 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T11:45:44.354 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T11:45:44.356 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T11:45:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T11:45:44.633 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T11:45:44.634 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:44.634 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:44.892 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-fuse 2024-09-21T11:45:45.209 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:45:45.210 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/2 2024-09-21T11:45:45.467 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T11:45:45.467 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:45.467 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:45.467 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T11:45:45.467 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:45.467 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:45.478 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:45.612 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 899 k 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:45.613 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:45.614 INFO:teuthology.orchestra.run.smithi193.stdout:Install 2 Packages 2024-09-21T11:45:45.614 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:45.614 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 979 k 2024-09-21T11:45:45.614 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 2.9 M 2024-09-21T11:45:45.614 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:45.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-volume 2024-09-21T11:45:45.752 INFO:teuthology.orchestra.run.smithi193.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 680 kB/s | 80 kB 00:00 2024-09-21T11:45:45.936 INFO:teuthology.orchestra.run.smithi193.stdout:(2/2): ceph-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_ 2.9 MB/s | 899 kB 00:00 2024-09-21T11:45:45.937 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:45.937 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.0 MB/s | 979 kB 00:00 2024-09-21T11:45:45.949 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:45.971 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:45.972 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:46.053 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:46.054 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:46.249 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:46.338 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:46.388 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T11:45:46.460 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:45:46.475 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 287 k 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:45:46.476 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:46.477 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 287 k 2024-09-21T11:45:46.477 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 1.4 M 2024-09-21T11:45:46.477 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:46.711 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-volume-19.3.0-5103.g5c1cdc3c.el9.noarch.rp 1.2 MB/s | 287 kB 00:00 2024-09-21T11:45:46.711 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:46.711 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.2 MB/s | 287 kB 00:00 2024-09-21T11:45:46.712 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:46.720 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:46.720 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:46.748 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:46.748 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T11:45:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:47.113 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:45:47.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/2 2024-09-21T11:45:47.390 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T11:45:47.391 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:47.391 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:47.391 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T11:45:47.391 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:47.391 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:47.779 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-volume 2024-09-21T11:45:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:48.027 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:48.027 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:48.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-09-21T11:45:48.381 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:32 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:48.515 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 287 k 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:45:48.516 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:48.517 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 287 k 2024-09-21T11:45:48.517 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 1.4 M 2024-09-21T11:45:48.517 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:48.750 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-volume-19.3.0-5103.g5c1cdc3c.el9.noarch.rp 1.2 MB/s | 287 kB 00:00 2024-09-21T11:45:48.751 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:48.751 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.2 MB/s | 287 kB 00:00 2024-09-21T11:45:48.751 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:48.760 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:48.760 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:48.788 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:48.789 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:48.938 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:49.006 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 126 k 2024-09-21T11:45:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 126 k 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 456 k 2024-09-21T11:45:49.072 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:49.077 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:49.094 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:49.094 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:45:49.094 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T11:45:49.094 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:49.292 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-19.3.0-5103.g5c1cdc3c.el9.x86_64 577 kB/s | 126 kB 00:00 2024-09-21T11:45:49.292 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:49.293 INFO:teuthology.orchestra.run.smithi107.stdout:Total 573 kB/s | 126 kB 00:00 2024-09-21T11:45:49.293 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:49.298 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:49.298 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:49.363 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:49.363 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:49.520 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:50.083 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:50.113 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:45:50.113 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:50.113 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:50.113 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:45:50.113 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:50.113 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:50.350 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:50.350 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:50.350 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:50.350 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:50.350 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:50.350 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:50.430 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados-devel 2024-09-21T11:45:50.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-09-21T11:45:51.028 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:51.162 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:51.162 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 126 k 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:45:51.163 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:51.164 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 126 k 2024-09-21T11:45:51.164 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 456 k 2024-09-21T11:45:51.164 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:51.202 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:39 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:51.340 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:51.341 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:51.341 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:51.373 INFO:teuthology.orchestra.run.smithi193.stdout:librados-devel-19.3.0-5103.g5c1cdc3c.el9.x86_64 606 kB/s | 126 kB 00:00 2024-09-21T11:45:51.373 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:51.374 INFO:teuthology.orchestra.run.smithi193.stdout:Total 602 kB/s | 126 kB 00:00 2024-09-21T11:45:51.374 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:51.379 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:51.379 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:51.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-09-21T11:45:51.443 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:51.443 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:51.598 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:51.703 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:51.986 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:39 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 32 k 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:52.120 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:45:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 32 k 2024-09-21T11:45:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 142 k 2024-09-21T11:45:52.121 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:52.123 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:52.398 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:52.398 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:52.398 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:52.398 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:52.398 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:52.398 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:52.438 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-19.3.0-5103.g5c1cdc3c.el9.x86_6 100 kB/s | 32 kB 00:00 2024-09-21T11:45:52.439 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:52.439 INFO:teuthology.orchestra.run.smithi107.stdout:Total 100 kB/s | 32 kB 00:00 2024-09-21T11:45:52.439 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:52.443 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:52.443 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:52.461 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:52.461 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:52.692 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs2 2024-09-21T11:45:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:53.230 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:45:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:53.231 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:53.286 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:53.379 INFO:teuthology.orchestra.run.smithi193.stdout:Package libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:53.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-09-21T11:45:53.425 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:53.426 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:53.426 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:53.508 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs-devel 2024-09-21T11:45:54.001 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:54.089 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:54.096 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:54.134 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:54.135 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:54.135 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:54.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-09-21T11:45:54.228 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 32 k 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:45:54.229 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:45:54.230 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:45:54.230 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:54.230 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 32 k 2024-09-21T11:45:54.230 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 142 k 2024-09-21T11:45:54.230 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:45:54.385 INFO:teuthology.orchestra.run.smithi193.stdout:libcephfs-devel-19.3.0-5103.g5c1cdc3c.el9.x86_6 204 kB/s | 32 kB 00:00 2024-09-21T11:45:54.386 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:54.386 INFO:teuthology.orchestra.run.smithi193.stdout:Total 202 kB/s | 32 kB 00:00 2024-09-21T11:45:54.386 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:45:54.390 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:45:54.390 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:45:54.405 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:45:54.406 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:45:54.494 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:45:54.591 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:54.785 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:54.876 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:54.920 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:54.921 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:54.921 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:54.975 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:54.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-09-21T11:45:55.214 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:55.214 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:55.214 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:45:55.214 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:45:55.214 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:45:55.214 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:55.443 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados2 2024-09-21T11:45:55.567 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:55.661 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:55.704 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:55.705 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:55.705 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:55.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-09-21T11:45:56.023 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:56.111 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:56.153 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:56.155 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:56.155 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:56.231 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librbd1 2024-09-21T11:45:56.354 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:56.444 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:56.486 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:56.546 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-09-21T11:45:56.820 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:56.909 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:56.953 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:56.954 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:56.954 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:57.043 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rados 2024-09-21T11:45:57.138 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:57.227 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:57.270 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:57.271 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:57.271 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:57.330 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-09-21T11:45:57.628 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:57.718 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:57.761 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:57.762 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:57.763 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:57.843 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rgw 2024-09-21T11:45:57.908 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:57.999 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:58.043 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:58.043 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-21T11:45:58.044 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:45:58.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-09-21T11:45:58.417 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:58.506 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:58.550 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:58.551 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:58.552 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:58.634 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-cephfs 2024-09-21T11:45:58.698 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:46 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:45:58.835 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:45:58.836 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:58.836 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-21T11:45:58.836 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:58.836 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:45:58.836 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 85 k 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 85 k 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 231 k 2024-09-21T11:45:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:45:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 434 kB/s | 85 kB 00:00 2024-09-21T11:45:59.034 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:45:59.035 INFO:teuthology.orchestra.run.smithi107.stdout:Total 431 kB/s | 85 kB 00:00 2024-09-21T11:45:59.035 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:45:59.042 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:45:59.042 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:45:59.108 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:45:59.108 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:45:59.217 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:45:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:45:59.306 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:45:59.351 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:45:59.352 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:45:59.352 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:45:59.400 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:45:59.434 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rbd 2024-09-21T11:45:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:00.017 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:46:00.074 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:00.074 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:00.074 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:46:00.074 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:46:00.075 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:00.075 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:46:00.106 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:46:00.150 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:46:00.151 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-09-21T11:46:00.152 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:46:00.239 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-fuse 2024-09-21T11:46:00.308 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-09-21T11:46:00.810 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:46:00.899 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:48 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 85 k 2024-09-21T11:46:00.942 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 85 k 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 231 k 2024-09-21T11:46:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:46:01.034 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:01.035 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 3.2 M 2024-09-21T11:46:01.036 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 14 M 2024-09-21T11:46:01.036 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:46:01.139 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 437 kB/s | 85 kB 00:00 2024-09-21T11:46:01.139 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:46:01.139 INFO:teuthology.orchestra.run.smithi193.stdout:Total 433 kB/s | 85 kB 00:00 2024-09-21T11:46:01.140 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:46:01.146 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:46:01.147 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:46:01.211 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:46:01.211 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:46:01.387 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 9.1 MB/s | 3.2 MB 00:00 2024-09-21T11:46:01.388 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:46:01.388 INFO:teuthology.orchestra.run.smithi107.stdout:Total 9.0 MB/s | 3.2 MB 00:00 2024-09-21T11:46:01.388 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:46:01.397 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:46:01.397 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:46:01.403 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:46:01.480 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:46:01.481 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:46:01.519 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:01.912 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:01.945 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:46:02.005 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:02.031 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:02.031 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:46:02.031 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T11:46:02.031 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:46:02.031 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:46:02.031 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:02.188 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:02.188 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:02.188 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:46:02.188 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:46:02.188 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:02.188 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:46:02.432 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-mirror 2024-09-21T11:46:03.020 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:47 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:46:03.096 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:03.096 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:03.096 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:46:03.096 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:46:03.097 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:03.097 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:46:03.156 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:03.157 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:46:03.158 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:03.158 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 3.2 M 2024-09-21T11:46:03.158 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 14 M 2024-09-21T11:46:03.158 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:46:03.324 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-09-21T11:46:03.501 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-mirror-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 9.3 MB/s | 3.2 MB 00:00 2024-09-21T11:46:03.501 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:46:03.502 INFO:teuthology.orchestra.run.smithi193.stdout:Total 9.3 MB/s | 3.2 MB 00:00 2024-09-21T11:46:03.502 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:46:03.511 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:46:03.511 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:46:03.593 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:46:03.593 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:46:03.917 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:51 ago on Sat 21 Sep 2024 11:44:12 AM UTC. 2024-09-21T11:46:04.032 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:46:04.048 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T11:46:04.049 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:46:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-21T11:46:04.049 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 170 k 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 170 k 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 490 k 2024-09-21T11:46:04.050 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T11:46:04.085 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:04.109 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:04.109 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:46:04.109 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T11:46:04.109 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:46:04.109 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:46:04.110 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:04.258 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 821 kB/s | 170 kB 00:00 2024-09-21T11:46:04.259 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:46:04.259 INFO:teuthology.orchestra.run.smithi107.stdout:Total 816 kB/s | 170 kB 00:00 2024-09-21T11:46:04.259 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T11:46:04.266 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T11:46:04.266 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T11:46:04.331 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T11:46:04.331 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T11:46:04.484 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T11:46:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:05.075 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:05.177 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:05.178 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:05.178 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:46:05.178 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:46:05.178 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:05.178 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:46:05.376 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:05.376 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:05.376 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T11:46:05.376 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:46:05.376 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:46:05.376 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T11:46:05.447 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-nbd 2024-09-21T11:46:05.599 DEBUG:teuthology.parallel:result is None 2024-09-21T11:46:06.050 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:50 ago on Sat 21 Sep 2024 11:44:16 AM UTC. 2024-09-21T11:46:06.183 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T11:46:06.184 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:06.184 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-21T11:46:06.184 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:06.184 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T11:46:06.184 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 170 k 2024-09-21T11:46:06.184 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 170 k 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 490 k 2024-09-21T11:46:06.185 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T11:46:06.388 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-nbd-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 837 kB/s | 170 kB 00:00 2024-09-21T11:46:06.389 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:46:06.389 INFO:teuthology.orchestra.run.smithi193.stdout:Total 830 kB/s | 170 kB 00:00 2024-09-21T11:46:06.390 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T11:46:06.398 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T11:46:06.398 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T11:46:06.466 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T11:46:06.466 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T11:46:06.622 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T11:46:06.723 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:07.115 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:07.399 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:46:07.399 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:07.399 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T11:46:07.399 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:46:07.399 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:46:07.399 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T11:46:07.676 DEBUG:teuthology.parallel:result is None 2024-09-21T11:46:07.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:46:07.818 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T11:46:07.842 INFO:teuthology.orchestra.run.smithi107.stdout:19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:46:07.843 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:46:07.843 INFO:teuthology.task.install:The correct ceph version 19.3.0-5103.g5c1cdc3c is installed. 2024-09-21T11:46:07.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:46:07.989 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T11:46:08.012 INFO:teuthology.orchestra.run.smithi193.stdout:19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:46:08.012 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:46:08.012 INFO:teuthology.task.install:The correct ceph version 19.3.0-5103.g5c1cdc3c is installed. 2024-09-21T11:46:08.014 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T11:46:08.014 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:46:08.014 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T11:46:08.044 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:46:08.044 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T11:46:08.079 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T11:46:08.079 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:46:08.079 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T11:46:08.113 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T11:46:08.181 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:46:08.181 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T11:46:08.207 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T11:46:08.273 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T11:46:08.273 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:46:08.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T11:46:08.300 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T11:46:08.366 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:46:08.366 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T11:46:08.391 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T11:46:08.457 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T11:46:08.457 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:46:08.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T11:46:08.483 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T11:46:08.548 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:46:08.548 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T11:46:08.573 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T11:46:08.638 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T11:46:08.730 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', 'CEPHADM_STRAY_DAEMON'], 'log-only-match': ['CEPHADM_'], 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'cephadm_mode': 'root'} 2024-09-21T11:46:08.730 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:46:08.730 INFO:tasks.cephadm:Cluster fsid is 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:46:08.730 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T11:46:08.731 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.c': '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]', 'mon.b': '172.21.15.193'} 2024-09-21T11:46:08.731 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-09-21T11:46:08.731 INFO:tasks.cephadm:First mgr is y 2024-09-21T11:46:08.731 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T11:46:08.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-09-21T11:46:08.757 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-21T11:46:08.782 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T11:46:08.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:46:08.928 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83200/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5103-g5c1cdc3c', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5103.g5c1cdc3c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 10:00:04.055809', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-21-0813', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-21T11:46:09.064 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-guits-main-2024-09-21-0813, sha1 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:46:09.065 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm 2024-09-21T11:46:09.065 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm 2024-09-21T11:46:09.065 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T11:46:09.453 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 804189 Sep 21 11:46 /home/ubuntu/cephtest/cephadm 2024-09-21T11:46:09.453 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T11:46:09.827 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 804189 Sep 21 11:46 /home/ubuntu/cephtest/cephadm 2024-09-21T11:46:09.828 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T11:46:09.845 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T11:46:09.874 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 on all hosts... 2024-09-21T11:46:09.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 pull 2024-09-21T11:46:09.888 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 pull 2024-09-21T11:46:11.489 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:46:11.507 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:47:09.002 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T11:47:09.003 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:47:09.003 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063", 2024-09-21T11:47:09.003 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-09-21T11:47:09.003 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472" 2024-09-21T11:47:09.003 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T11:47:09.003 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T11:47:34.291 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T11:47:34.291 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:47:34.291 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063", 2024-09-21T11:47:34.291 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-21T11:47:34.291 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472" 2024-09-21T11:47:34.292 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-21T11:47:34.292 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T11:47:34.317 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-09-21T11:47:34.349 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-21T11:47:34.376 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-09-21T11:47:34.419 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-21T11:47:34.443 INFO:tasks.cephadm:Writing seed config... 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-21T11:47:34.444 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T11:47:34.445 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T11:47:34.445 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-21T11:47:34.445 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-21T11:47:34.445 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T11:47:34.445 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:47:34.445 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T11:47:34.476 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1795c04c-780f-11ef-baf6-efdc52797490 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-21T11:47:34.476 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service 2024-09-21T11:47:34.518 DEBUG:teuthology.orchestra.run.smithi107:mgr.y> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y.service 2024-09-21T11:47:34.563 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T11:47:34.563 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 -v bootstrap --fsid 1795c04c-780f-11ef-baf6-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T11:47:34.770 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:47:34.770 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301', '-v', 'bootstrap', '--fsid', '1795c04c-780f-11ef-baf6-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-09-21T11:47:34.770 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T11:47:34.771 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-09-21T11:47:34.796 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-21T11:47:34.796 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-09-21T11:47:34.796 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-09-21T11:47:34.804 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T11:47:34.804 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T11:47:34.810 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T11:47:34.811 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-21T11:47:34.818 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-09-21T11:47:34.824 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-09-21T11:47:34.824 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-09-21T11:47:34.825 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-09-21T11:47:34.847 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-21T11:47:34.848 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T11:47:34.848 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-09-21T11:47:34.848 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-09-21T11:47:34.854 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T11:47:34.854 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T11:47:34.860 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T11:47:34.860 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-21T11:47:34.867 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140473002789232 on /run/cephadm/1795c04c-780f-11ef-baf6-efdc52797490.lock 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140473002789232 acquired on /run/cephadm/1795c04c-780f-11ef-baf6-efdc52797490.lock 2024-09-21T11:47:34.874 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-09-21T11:47:34.875 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-09-21T11:47:34.875 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-21T11:47:34.878 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-09-21T11:47:34.878 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-09-21T11:47:34.880 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T11:47:34.880 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T11:47:34.880 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T11:47:34.883 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T11:47:34.883 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T11:47:34.883 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T11:47:34.883 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T11:47:34.883 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link noprefixroute 2024-09-21T11:47:34.883 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T11:47:34.884 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-09-21T11:47:34.884 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-09-21T11:47:34.884 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T11:47:34.884 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T11:47:34.885 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:832cb702be6ace10bd8c779318aa247eb0f26450f5c31f0789905499e59c463e 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 2024-09-21T11:47:35.352 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T11:47:42.102 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev) 2024-09-21T11:47:42.102 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev) 2024-09-21T11:47:42.102 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T11:47:51.116 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-09-21T11:47:51.116 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-09-21T11:47:52.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBnsu5mCmJBHBAAKe1D7j2F+1yYwcUsRHkFJQ== 2024-09-21T11:47:53.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBosu5mGacBHxAAkj9eDJtm/V50USMIKUCJvQ== 2024-09-21T11:47:54.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBpsu5mNHvKFhAAvY6N7BzUaB36lWe8BOmneg== 2024-09-21T11:47:54.080 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = quincy 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T11:47:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:47:54.986 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-09-21T11:47:55.924 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-09-21T11:47:56.107 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T11:47:56.246 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T11:47:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1795c04c-780f-11ef-baf6-efdc52797490.target → /etc/systemd/system/ceph-1795c04c-780f-11ef-baf6-efdc52797490.target. 2024-09-21T11:47:56.435 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1795c04c-780f-11ef-baf6-efdc52797490.target → /etc/systemd/system/ceph-1795c04c-780f-11ef-baf6-efdc52797490.target. 2024-09-21T11:47:56.655 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a 2024-09-21T11:47:56.655 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service: Unit ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service not loaded. 2024-09-21T11:47:56.808 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1795c04c-780f-11ef-baf6-efdc52797490.target.wants/ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service → /etc/systemd/system/ceph-1795c04c-780f-11ef-baf6-efdc52797490@.service. 2024-09-21T11:47:57.303 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:57 smithi107 podman[29787]: 2024-09-21 11:47:57.262065957 +0000 UTC m=+0.334453297 container start 33380d7ec9bbc2edee649373bde357b12b0a52125aaaf4268a4face2832bc9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T11:47:57.356 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:47:57.356 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-21T11:47:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:47:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-21T11:47:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-21T11:47:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T11:47:57.364 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-09-21T11:47:57.364 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-09-21T11:47:57.592 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:57 smithi107 ceph-mon[29823]: mkfs 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:57.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:57 smithi107 ceph-mon[29823]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T11:47:57.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:57 smithi107 bash[29787]: 33380d7ec9bbc2edee649373bde357b12b0a52125aaaf4268a4face2832bc9bc 2024-09-21T11:47:57.593 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:57 smithi107 systemd[1]: Started Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:47:58.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: monmap epoch 1 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: last_changed 2024-09-21T11:47:54.410883+0000 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: min_mon_release 19 (squid) 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: election_strategy: 1 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: fsmap 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: mgrmap e1: no daemons active 2024-09-21T11:47:58.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:58 smithi107 ceph-mon[29823]: from='client.? 172.21.15.107:0/2929162165' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T11:47:58.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T11:47:58.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:58.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T11:47:58.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:47:58.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-09-21T11:47:58.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.562559s) 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.789+0000 7fdf5332f640 1 Processor -- start 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.789+0000 7fdf5332f640 1 -- start start 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.789+0000 7fdf5332f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.789+0000 7fdf5332f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c107000 con 0x7fdf4c106610 2024-09-21T11:47:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.790+0000 7fdf510cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.790+0000 7fdf510cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44712/0 (socket says 172.21.15.107:44712) 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.790+0000 7fdf510cb640 1 -- 172.21.15.107:0/1294195573 learned_addr learned my addr 172.21.15.107:0/1294195573 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.790+0000 7fdf510cb640 1 -- 172.21.15.107:0/1294195573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c107180 con 0x7fdf4c106610 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf510cb640 1 --2- 172.21.15.107:0/1294195573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009820 tx=0x7fdf4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e9d3c608f94527 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf4bfff640 1 -- 172.21.15.107:0/1294195573 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44037070 con 0x7fdf4c106610 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf4bfff640 1 -- 172.21.15.107:0/1294195573 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf44032440 con 0x7fdf4c106610 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf4bfff640 1 -- 172.21.15.107:0/1294195573 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44032740 con 0x7fdf4c106610 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 -- 172.21.15.107:0/1294195573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 msgr2=0x7fdf4c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 --2- 172.21.15.107:0/1294195573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009820 tx=0x7fdf4402a270 comp rx=0 tx=0).stop 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 -- 172.21.15.107:0/1294195573 shutdown_connections 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 --2- 172.21.15.107:0/1294195573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c106a10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 -- 172.21.15.107:0/1294195573 >> 172.21.15.107:0/1294195573 conn(0x7fdf4c101f00 msgr2=0x7fdf4c104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:47:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 -- 172.21.15.107:0/1294195573 shutdown_connections 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.792+0000 7fdf5332f640 1 -- 172.21.15.107:0/1294195573 wait complete. 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf5332f640 1 Processor -- start 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf5332f640 1 -- start start 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf5332f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf5332f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c107b90 con 0x7fdf4c106610 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf510cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf510cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44716/0 (socket says 172.21.15.107:44716) 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.793+0000 7fdf510cb640 1 -- 172.21.15.107:0/2929162165 learned_addr learned my addr 172.21.15.107:0/2929162165 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf510cb640 1 -- 172.21.15.107:0/2929162165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c19a4c0 con 0x7fdf4c106610 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf510cb640 1 --2- 172.21.15.107:0/2929162165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c199f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf440090b0 tx=0x7fdf440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44041070 con 0x7fdf4c106610 2024-09-21T11:47:58.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4403c7a0 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf44037040 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c19a750 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c19b110 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf44044070 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.794+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf4403d3f0 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.795+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c108340 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.797+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdf44046450 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.858+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdf4c19b690 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.859+0000 7fdf4a7fc640 1 -- 172.21.15.107:0/2929162165 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdf44046630 con 0x7fdf4c106610 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 msgr2=0x7fdf4c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:47:58.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 --2- 172.21.15.107:0/2929162165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c199f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf440090b0 tx=0x7fdf440046c0 comp rx=0 tx=0).stop 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 shutdown_connections 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 --2- 172.21.15.107:0/2929162165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf4c106610 0x7fdf4c199f60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 >> 172.21.15.107:0/2929162165 conn(0x7fdf4c101f00 msgr2=0x7fdf4c18e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 shutdown_connections 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:57.860+0000 7fdf5332f640 1 -- 172.21.15.107:0/2929162165 wait complete. 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-09-21T11:47:58.411 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T11:47:59.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:59 smithi107 ceph-mon[29823]: from='client.? 172.21.15.107:0/2301286079' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T11:47:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:47:59 smithi107 ceph-mon[29823]: from='client.? 172.21.15.107:0/2301286079' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T11:47:59.506 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T11:47:59.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b74df640 1 Processor -- start 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b74df640 1 -- start start 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b74df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b0105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b74df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0105f00 con 0x7ff5b0105510 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b527b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b0105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b527b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b0105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44724/0 (socket says 172.21.15.107:44724) 2024-09-21T11:47:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b527b640 1 -- 172.21.15.107:0/4294285106 learned_addr learned my addr 172.21.15.107:0/4294285106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.867+0000 7ff5b527b640 1 -- 172.21.15.107:0/4294285106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b0106080 con 0x7ff5b0105510 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.868+0000 7ff5b527b640 1 --2- 172.21.15.107:0/4294285106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b0105910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009820 tx=0x7ff5a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce92acacc4db5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.868+0000 7ff5a7fff640 1 -- 172.21.15.107:0/4294285106 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0037070 con 0x7ff5b0105510 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.868+0000 7ff5a7fff640 1 -- 172.21.15.107:0/4294285106 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a0032440 con 0x7ff5b0105510 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.868+0000 7ff5a7fff640 1 -- 172.21.15.107:0/4294285106 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0032740 con 0x7ff5b0105510 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 -- 172.21.15.107:0/4294285106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 msgr2=0x7ff5b0105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 --2- 172.21.15.107:0/4294285106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b0105910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009820 tx=0x7ff5a002a270 comp rx=0 tx=0).stop 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 -- 172.21.15.107:0/4294285106 shutdown_connections 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 --2- 172.21.15.107:0/4294285106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b0105910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 -- 172.21.15.107:0/4294285106 >> 172.21.15.107:0/4294285106 conn(0x7ff5b0100e00 msgr2=0x7ff5b0103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 -- 172.21.15.107:0/4294285106 shutdown_connections 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 -- 172.21.15.107:0/4294285106 wait complete. 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.869+0000 7ff5b74df640 1 Processor -- start 2024-09-21T11:47:59.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b74df640 1 -- start start 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b74df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b007a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b74df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0106a90 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b527b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b007a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b527b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b007a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44736/0 (socket says 172.21.15.107:44736) 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b527b640 1 -- 172.21.15.107:0/2301286079 learned_addr learned my addr 172.21.15.107:0/2301286079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b527b640 1 -- 172.21.15.107:0/2301286079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b007a5e0 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b527b640 1 --2- 172.21.15.107:0/2301286079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b007a080 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00090b0 tx=0x7ff5a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0041070 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a003c7a0 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b019cc50 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.870+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a0037040 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.871+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b019d130 con 0x7ff5b0105510 2024-09-21T11:47:59.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.871+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff5a0044070 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.871+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5a003d3f0 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.872+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0107240 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.874+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff5a0046450 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.931+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff5b019d470 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.933+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+376 (secure 0 0 0) 0x7ff5a0046630 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.934+0000 7ff5a67fc640 1 -- 172.21.15.107:0/2301286079 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7ff5a0058480 con 0x7ff5b0105510 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 msgr2=0x7ff5b007a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 --2- 172.21.15.107:0/2301286079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b007a080 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00090b0 tx=0x7ff5a00046c0 comp rx=0 tx=0).stop 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 shutdown_connections 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 --2- 172.21.15.107:0/2301286079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0105510 0x7ff5b007a080 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 >> 172.21.15.107:0/2301286079 conn(0x7ff5b0100e00 msgr2=0x7ff5b018d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 shutdown_connections 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:58.935+0000 7ff5b74df640 1 -- 172.21.15.107:0/2301286079 wait complete. 2024-09-21T11:47:59.516 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-09-21T11:48:00.585 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:00 smithi107 ceph-mon[29823]: from='client.? 172.21.15.107:0/2954102872' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:48:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe175966640 1 Processor -- start 2024-09-21T11:48:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe175966640 1 -- start start 2024-09-21T11:48:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe175966640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe170105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:00.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe175966640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe170105f00 con 0x7fe170105510 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe16effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe170105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe16effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe170105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44748/0 (socket says 172.21.15.107:44748) 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.952+0000 7fe16effd640 1 -- 172.21.15.107:0/3604441772 learned_addr learned my addr 172.21.15.107:0/3604441772 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.953+0000 7fe16effd640 1 -- 172.21.15.107:0/3604441772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe170106080 con 0x7fe170105510 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.953+0000 7fe16effd640 1 --2- 172.21.15.107:0/3604441772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe170105910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe160009e80 tx=0x7fe16002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cc0064a52cdacb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.953+0000 7fe16dffb640 1 -- 172.21.15.107:0/3604441772 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160036070 con 0x7fe170105510 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.953+0000 7fe16dffb640 1 -- 172.21.15.107:0/3604441772 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe160031830 con 0x7fe170105510 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.953+0000 7fe16dffb640 1 -- 172.21.15.107:0/3604441772 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160007780 con 0x7fe170105510 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 -- 172.21.15.107:0/3604441772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 msgr2=0x7fe170105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 --2- 172.21.15.107:0/3604441772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe170105910 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe160009e80 tx=0x7fe16002a7a0 comp rx=0 tx=0).stop 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 -- 172.21.15.107:0/3604441772 shutdown_connections 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 --2- 172.21.15.107:0/3604441772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe170105910 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 -- 172.21.15.107:0/3604441772 >> 172.21.15.107:0/3604441772 conn(0x7fe170100e00 msgr2=0x7fe170103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 -- 172.21.15.107:0/3604441772 shutdown_connections 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.954+0000 7fe175966640 1 -- 172.21.15.107:0/3604441772 wait complete. 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe175966640 1 Processor -- start 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe175966640 1 -- start start 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe175966640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe17019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe175966640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe170109990 con 0x7fe170105510 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe16effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe17019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe16effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe17019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44764/0 (socket says 172.21.15.107:44764) 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe16effd640 1 -- 172.21.15.107:0/2954102872 learned_addr learned my addr 172.21.15.107:0/2954102872 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.955+0000 7fe16effd640 1 -- 172.21.15.107:0/2954102872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17019abb0 con 0x7fe170105510 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe16effd640 1 --2- 172.21.15.107:0/2954102872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe17019a650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe160009fa0 tx=0x7fe1600077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160041020 con 0x7fe170105510 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe16003c470 con 0x7fe170105510 2024-09-21T11:48:00.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17019ae40 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe160036040 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17019b880 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe160044020 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.956+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe160046400 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.957+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170109b10 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:47:59.959+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe16004b020 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.017+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe17019be40 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.017+0000 7fe174964640 1 -- 172.21.15.107:0/2954102872 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fe16003c900 con 0x7fe170105510 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 msgr2=0x7fe17019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 --2- 172.21.15.107:0/2954102872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe17019a650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe160009fa0 tx=0x7fe1600077a0 comp rx=0 tx=0).stop 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 shutdown_connections 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 --2- 172.21.15.107:0/2954102872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe170105510 0x7fe17019a650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 >> 172.21.15.107:0/2954102872 conn(0x7fe170100e00 msgr2=0x7fe170079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 shutdown_connections 2024-09-21T11:48:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:00.018+0000 7fe175966640 1 -- 172.21.15.107:0/2954102872 wait complete. 2024-09-21T11:48:00.595 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-09-21T11:48:00.857 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:00 smithi107 systemd[1]: Stopping Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:48:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a[29801]: 2024-09-21T11:48:00.854+0000 7f6ad7d22640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T11:48:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a[29801]: 2024-09-21T11:48:00.854+0000 7f6ad7d22640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T11:48:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:00 smithi107 podman[30140]: 2024-09-21 11:48:00.988409865 +0000 UTC m=+0.242725317 container died 33380d7ec9bbc2edee649373bde357b12b0a52125aaaf4268a4face2832bc9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T11:48:01.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 podman[30140]: 2024-09-21 11:48:01.344278124 +0000 UTC m=+0.598593575 container cleanup 33380d7ec9bbc2edee649373bde357b12b0a52125aaaf4268a4face2832bc9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T11:48:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 bash[30140]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a 2024-09-21T11:48:01.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 podman[30159]: 2024-09-21 11:48:01.586595538 +0000 UTC m=+0.593897318 container remove 33380d7ec9bbc2edee649373bde357b12b0a52125aaaf4268a4face2832bc9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T11:48:01.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T11:48:01.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 systemd[1]: Stopped Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:48:01.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 systemd[1]: Starting Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:48:02.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 podman[30255]: 2024-09-21 11:48:01.895755162 +0000 UTC m=+0.085496457 container create 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T11:48:02.150 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:01 smithi107 podman[30255]: 2024-09-21 11:48:01.825693306 +0000 UTC m=+0.015434618 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:48:02.151 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 podman[30255]: 2024-09-21 11:48:02.123233726 +0000 UTC m=+0.312975031 container init 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T11:48:02.151 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 podman[30255]: 2024-09-21 11:48:02.128031544 +0000 UTC m=+0.317772840 container start 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T11:48:02.212 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev), process ceph-mon, pid 7 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: pidfile_write: ignore empty --pid-file 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: load: jerasure load: lrc 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: RocksDB version: 7.9.2 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Git sha 0 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Compile date 2024-09-21 08:36:07 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: DB SUMMARY 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: DB Session ID: 2CP0K8BW5CQD06XMOT9P 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: CURRENT file: CURRENT 2024-09-21T11:48:02.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 91036 ; 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.error_if_exists: 0 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.create_if_missing: 0 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.paranoid_checks: 1 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.env: 0x556b6ca38120 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.info_log: 0x556b6d43d700 2024-09-21T11:48:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.statistics: (nil) 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.use_fsync: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_log_file_size: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.allow_fallocate: 1 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.use_direct_reads: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T11:48:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.db_log_dir: 2024-09-21T11:48:02.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.wal_dir: 2024-09-21T11:48:02.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.write_buffer_manager: 0x556b6d441b80 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T11:48:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.unordered_write: 0 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.row_cache: None 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.wal_filter: None 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T11:48:02.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.two_write_queues: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.wal_compression: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.atomic_flush: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.log_readahead_size: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T11:48:02.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_background_jobs: 2 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_background_compactions: -1 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_subcompactions: 1 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_open_files: -1 2024-09-21T11:48:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_background_flushes: -1 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Compression algorithms supported: 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kZSTD supported: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kXpressCompression supported: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kBZip2Compression supported: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kLZ4Compression supported: 1 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kZlibCompression supported: 1 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: kSnappyCompression supported: 1 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T11:48:02.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.merge_operator: 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_filter: None 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556b6d43d320) 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T11:48:02.473 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x556b6d431610 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-09-21T11:48:02.474 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-09-21T11:48:02.475 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression: NoCompression 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.num_levels: 7 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T11:48:02.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:48:02.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T11:48:02.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T11:48:02.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.inplace_update_support: 0 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.bloom_locality: 0 2024-09-21T11:48:02.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.max_successive_merges: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.ttl: 2592000 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.enable_blob_files: false 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.min_blob_size: 0 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T11:48:02.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4424fb34-a5e6-4481-934d-6741343a771a 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726919282154767, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726919282155625, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 87501, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 269, "table_properties": {"data_size": 85649, "index_size": 236, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10710, "raw_average_key_size": 46, "raw_value_size": 79553, "raw_average_value_size": 348, "num_data_blocks": 11, "num_entries": 228, "num_filter_entries": 228, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726919282, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4424fb34-a5e6-4481-934d-6741343a771a", "db_session_id": "2CP0K8BW5CQD06XMOT9P", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726919282155718, "job": 1, "event": "recovery_finished"} 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T11:48:02.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556b6d460e00 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: DB pointer 0x556b6d562000 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:48:02.483 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 87.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 87.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 152.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 152.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T11:48:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 28.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 28.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x556b6d431610#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???) e1 preinit fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).mds e1 new map 2024-09-21T11:48:02.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).mds e1 print_map 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: btime 2024-09-21T11:47:57:299923+0000 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:48:02.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 bash[30255]: 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 systemd[1]: Started Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: monmap epoch 1 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: last_changed 2024-09-21T11:47:54.410883+0000 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: min_mon_release 19 (squid) 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: election_strategy: 1 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: fsmap 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T11:48:02.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:02 smithi107 ceph-mon[30293]: mgrmap e1: no daemons active 2024-09-21T11:48:03.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.658+0000 7f7ee3885640 1 Processor -- start 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.658+0000 7f7ee3885640 1 -- start start 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.658+0000 7f7ee3885640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.658+0000 7f7ee3885640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc105f00 con 0x7f7edc105510 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.659+0000 7f7ee1621640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.659+0000 7f7ee1621640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44778/0 (socket says 172.21.15.107:44778) 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.659+0000 7f7ee1621640 1 -- 172.21.15.107:0/673780337 learned_addr learned my addr 172.21.15.107:0/673780337 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.659+0000 7f7ee1621640 1 -- 172.21.15.107:0/673780337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc106080 con 0x7f7edc105510 2024-09-21T11:48:03.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.660+0000 7f7ee1621640 1 --2- 172.21.15.107:0/673780337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc105910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009820 tx=0x7f7ed802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce4af38dfc73df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ed3fff640 1 -- 172.21.15.107:0/673780337 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed8037070 con 0x7f7edc105510 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ed3fff640 1 -- 172.21.15.107:0/673780337 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f7ed8032870 con 0x7f7edc105510 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ed3fff640 1 -- 172.21.15.107:0/673780337 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed80305d0 con 0x7f7edc105510 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 -- 172.21.15.107:0/673780337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 msgr2=0x7f7edc105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 --2- 172.21.15.107:0/673780337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc105910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009820 tx=0x7f7ed802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 -- 172.21.15.107:0/673780337 shutdown_connections 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 --2- 172.21.15.107:0/673780337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc105910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 -- 172.21.15.107:0/673780337 >> 172.21.15.107:0/673780337 conn(0x7f7edc100e00 msgr2=0x7f7edc103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 -- 172.21.15.107:0/673780337 shutdown_connections 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.661+0000 7f7ee3885640 1 -- 172.21.15.107:0/673780337 wait complete. 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee3885640 1 Processor -- start 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee3885640 1 -- start start 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee3885640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee3885640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc109990 con 0x7f7edc105510 2024-09-21T11:48:03.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee1621640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee1621640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44782/0 (socket says 172.21.15.107:44782) 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee1621640 1 -- 172.21.15.107:0/3923699952 learned_addr learned my addr 172.21.15.107:0/3923699952 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.662+0000 7f7ee1621640 1 -- 172.21.15.107:0/3923699952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc19abc0 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ee1621640 1 --2- 172.21.15.107:0/3923699952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc19a660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80090b0 tx=0x7f7ed80307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed803f020 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f7ed8030e20 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc19ae50 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ed8037040 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc19b810 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ed8042020 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.663+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ed8044930 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.664+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc109b10 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.666+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7ed8049020 con 0x7f7edc105510 2024-09-21T11:48:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.724+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7edc105990 con 0x7f7edc105510 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.726+0000 7f7ed27fc640 1 -- 172.21.15.107:0/3923699952 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7ed8044400 con 0x7f7edc105510 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.726+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 msgr2=0x7f7edc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.726+0000 7f7ee3885640 1 --2- 172.21.15.107:0/3923699952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc19a660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80090b0 tx=0x7f7ed80307c0 comp rx=0 tx=0).stop 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.727+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 shutdown_connections 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.727+0000 7f7ee3885640 1 --2- 172.21.15.107:0/3923699952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7edc105510 0x7f7edc19a660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.727+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 >> 172.21.15.107:0/3923699952 conn(0x7f7edc100e00 msgr2=0x7f7edc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.727+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 shutdown_connections 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:02.727+0000 7f7ee3885640 1 -- 172.21.15.107:0/3923699952 wait complete. 2024-09-21T11:48:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T11:48:03.221 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T11:48:03.221 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-09-21T11:48:03.221 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T11:48:03.222 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T11:48:03.479 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y 2024-09-21T11:48:03.479 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y.service: Unit ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y.service not loaded. 2024-09-21T11:48:03.651 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1795c04c-780f-11ef-baf6-efdc52797490.target.wants/ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y.service → /etc/systemd/system/ceph-1795c04c-780f-11ef-baf6-efdc52797490@.service. 2024-09-21T11:48:03.673 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:03 smithi107 systemd[1]: Starting Ceph mgr.y for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:48:03.993 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3923699952' entity='client.admin' 2024-09-21T11:48:03.994 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:03 smithi107 podman[30526]: 2024-09-21 11:48:03.880816896 +0000 UTC m=+0.111385561 container create e60ab524a9f6ad9aac3f0a69d75f152cf14152b50b5aca11829b16f86b2295cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T11:48:03.994 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:03 smithi107 podman[30526]: 2024-09-21 11:48:03.785059316 +0000 UTC m=+0.015627996 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:48:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:48:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-21T11:48:04.234 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:48:04.234 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-21T11:48:04.234 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-21T11:48:04.234 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T11:48:04.244 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:48:04.244 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-21T11:48:04.253 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:48:04.253 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-21T11:48:04.253 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-21T11:48:04.253 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T11:48:04.253 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-09-21T11:48:04.253 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-09-21T11:48:04.376 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:04 smithi107 podman[30526]: 2024-09-21 11:48:04.099655122 +0000 UTC m=+0.330223786 container init e60ab524a9f6ad9aac3f0a69d75f152cf14152b50b5aca11829b16f86b2295cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T11:48:04.377 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:04 smithi107 podman[30526]: 2024-09-21 11:48:04.104456089 +0000 UTC m=+0.335024748 container start e60ab524a9f6ad9aac3f0a69d75f152cf14152b50b5aca11829b16f86b2295cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T11:48:04.377 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:04 smithi107 bash[30526]: e60ab524a9f6ad9aac3f0a69d75f152cf14152b50b5aca11829b16f86b2295cf 2024-09-21T11:48:04.377 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:04 smithi107 systemd[1]: Started Ceph mgr.y for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:48:05.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2396336924' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T11:48:05.370 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:48:05.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:05.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1795c04c-780f-11ef-baf6-efdc52797490", 2024-09-21T11:48:05.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T11:48:05.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T11:48:05.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T11:48:05.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T11:48:05.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:05.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T11:48:05.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T11:48:05.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T11:48:05.381 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T11:47:57:299923+0000", 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T11:48:05.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T11:47:57.300583+0000", 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.724+0000 7f6a935d0640 1 Processor -- start 2024-09-21T11:48:05.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.724+0000 7f6a935d0640 1 -- start start 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.724+0000 7f6a935d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.724+0000 7f6a935d0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1096c0 con 0x7f6a8c079bc0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.725+0000 7f6a9136c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.725+0000 7f6a9136c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48682/0 (socket says 172.21.15.107:48682) 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.725+0000 7f6a9136c640 1 -- 172.21.15.107:0/3260912806 learned_addr learned my addr 172.21.15.107:0/3260912806 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.725+0000 7f6a9136c640 1 -- 172.21.15.107:0/3260912806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c109840 con 0x7f6a8c079bc0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.726+0000 7f6a9136c640 1 --2- 172.21.15.107:0/3260912806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c109180 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009820 tx=0x7f6a8802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f414cac931e29ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.726+0000 7f6a83fff640 1 -- 172.21.15.107:0/3260912806 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a88037070 con 0x7f6a8c079bc0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.726+0000 7f6a83fff640 1 -- 172.21.15.107:0/3260912806 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f6a88032870 con 0x7f6a8c079bc0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.726+0000 7f6a83fff640 1 -- 172.21.15.107:0/3260912806 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a88030660 con 0x7f6a8c079bc0 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 -- 172.21.15.107:0/3260912806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 msgr2=0x7f6a8c109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 --2- 172.21.15.107:0/3260912806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c109180 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009820 tx=0x7f6a8802a300 comp rx=0 tx=0).stop 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 -- 172.21.15.107:0/3260912806 shutdown_connections 2024-09-21T11:48:05.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 --2- 172.21.15.107:0/3260912806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c109180 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 -- 172.21.15.107:0/3260912806 >> 172.21.15.107:0/3260912806 conn(0x7f6a8c100df0 msgr2=0x7f6a8c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 -- 172.21.15.107:0/3260912806 shutdown_connections 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 -- 172.21.15.107:0/3260912806 wait complete. 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.727+0000 7f6a935d0640 1 Processor -- start 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a935d0640 1 -- start start 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a935d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a935d0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c10a8b0 con 0x7f6a8c079bc0 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a9136c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a9136c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48684/0 (socket says 172.21.15.107:48684) 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a9136c640 1 -- 172.21.15.107:0/2396336924 learned_addr learned my addr 172.21.15.107:0/2396336924 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a9136c640 1 -- 172.21.15.107:0/2396336924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c19abf0 con 0x7f6a8c079bc0 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a9136c640 1 --2- 172.21.15.107:0/2396336924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c19a690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a88037040 con 0x7f6a8c079bc0 2024-09-21T11:48:05.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c19ae80 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f6a88040070 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a8803a390 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.728+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c19b840 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.729+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a8803a610 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.729+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a88045870 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.729+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8c1071f0 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.731+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6a88030d50 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.793+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6a8c19baa0 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.794+0000 7f6a827fc640 1 -- 172.21.15.107:0/2396336924 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f6a88045a70 con 0x7f6a8c079bc0 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 msgr2=0x7f6a8c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 --2- 172.21.15.107:0/2396336924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c19a690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6a88000c00 tx=0x7f6a88030a60 comp rx=0 tx=0).stop 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 shutdown_connections 2024-09-21T11:48:05.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 --2- 172.21.15.107:0/2396336924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c079bc0 0x7f6a8c19a690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:05.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 >> 172.21.15.107:0/2396336924 conn(0x7f6a8c100df0 msgr2=0x7f6a8c101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:05.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 shutdown_connections 2024-09-21T11:48:05.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:04.795+0000 7f6a935d0640 1 -- 172.21.15.107:0/2396336924 wait complete. 2024-09-21T11:48:05.387 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-09-21T11:48:08.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/65678373' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T11:48:08.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:48:08.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1795c04c-780f-11ef-baf6-efdc52797490", 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T11:48:08.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T11:48:08.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T11:48:08.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T11:48:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T11:48:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T11:47:57:299923+0000", 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T11:48:08.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T11:47:57.300583+0000", 2024-09-21T11:48:08.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.822+0000 7fe8f465f640 1 Processor -- start 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.822+0000 7fe8f465f640 1 -- start start 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.823+0000 7fe8f465f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec1091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.823+0000 7fe8f465f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec109720 con 0x7fe8ec079b60 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.823+0000 7fe8f23fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec1091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.823+0000 7fe8f23fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec1091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48692/0 (socket says 172.21.15.107:48692) 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.823+0000 7fe8f23fb640 1 -- 172.21.15.107:0/1123278305 learned_addr learned my addr 172.21.15.107:0/1123278305 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.823+0000 7fe8f23fb640 1 -- 172.21.15.107:0/1123278305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec1098a0 con 0x7fe8ec079b60 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.824+0000 7fe8f23fb640 1 --2- 172.21.15.107:0/1123278305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec1091e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009820 tx=0x7fe8dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da159a081815987 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.824+0000 7fe8f13f9640 1 -- 172.21.15.107:0/1123278305 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8dc037070 con 0x7fe8ec079b60 2024-09-21T11:48:08.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.824+0000 7fe8f13f9640 1 -- 172.21.15.107:0/1123278305 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe8dc032870 con 0x7fe8ec079b60 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.824+0000 7fe8f13f9640 1 -- 172.21.15.107:0/1123278305 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8dc0305d0 con 0x7fe8ec079b60 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 -- 172.21.15.107:0/1123278305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 msgr2=0x7fe8ec1091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 --2- 172.21.15.107:0/1123278305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec1091e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009820 tx=0x7fe8dc02a270 comp rx=0 tx=0).stop 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 -- 172.21.15.107:0/1123278305 shutdown_connections 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 --2- 172.21.15.107:0/1123278305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec1091e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 -- 172.21.15.107:0/1123278305 >> 172.21.15.107:0/1123278305 conn(0x7fe8ec100df0 msgr2=0x7fe8ec103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 -- 172.21.15.107:0/1123278305 shutdown_connections 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.825+0000 7fe8f465f640 1 -- 172.21.15.107:0/1123278305 wait complete. 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f465f640 1 Processor -- start 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f465f640 1 -- start start 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f465f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f465f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec10a840 con 0x7fe8ec079b60 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f23fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:08.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f23fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48698/0 (socket says 172.21.15.107:48698) 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f23fb640 1 -- 172.21.15.107:0/65678373 learned_addr learned my addr 172.21.15.107:0/65678373 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f23fb640 1 -- 172.21.15.107:0/65678373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec19ab80 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f23fb640 1 --2- 172.21.15.107:0/65678373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec19a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0097f0 tx=0x7fe8dc030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8dc03f020 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe8dc030dc0 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8dc037040 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.826+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec19ae10 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.827+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec19b850 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.827+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8dc042020 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.827+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe8dc0448e0 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.827+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ec107250 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.829+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe8dc049020 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.892+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe8ec19be00 con 0x7fe8ec079b60 2024-09-21T11:48:08.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.892+0000 7fe8e37fe640 1 -- 172.21.15.107:0/65678373 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe8dc044400 con 0x7fe8ec079b60 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 msgr2=0x7fe8ec19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 --2- 172.21.15.107:0/65678373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec19a620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0097f0 tx=0x7fe8dc030b00 comp rx=0 tx=0).stop 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 shutdown_connections 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 --2- 172.21.15.107:0/65678373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8ec079b60 0x7fe8ec19a620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 >> 172.21.15.107:0/65678373 conn(0x7fe8ec100df0 msgr2=0x7fe8ec101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 shutdown_connections 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:07.894+0000 7fe8f465f640 1 -- 172.21.15.107:0/65678373 wait complete. 2024-09-21T11:48:08.468 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: Activating manager daemon y 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: mgrmap e2: y(active, starting, since 0.00329905s) 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: Manager daemon y is now available 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' 2024-09-21T11:48:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' 2024-09-21T11:48:09.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:09 smithi107 ceph-mon[30293]: from='mgr.14100 172.21.15.107:0/3035093189' entity='mgr.y' 2024-09-21T11:48:11.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:11 smithi107 ceph-mon[30293]: mgrmap e3: y(active, since 1.00718s) 2024-09-21T11:48:11.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1638695086' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T11:48:11.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1795c04c-780f-11ef-baf6-efdc52797490", 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:11.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T11:48:11.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T11:48:11.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T11:48:11.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T11:47:57:299923+0000", 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T11:48:11.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T11:47:57.300583+0000", 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:48:11.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc30067640 1 Processor -- start 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc30067640 1 -- start start 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc30067640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc28106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc30067640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28107000 con 0x7fdc28106610 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc2de03640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc28106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc2de03640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc28106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48752/0 (socket says 172.21.15.107:48752) 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.899+0000 7fdc2de03640 1 -- 172.21.15.107:0/584457956 learned_addr learned my addr 172.21.15.107:0/584457956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.900+0000 7fdc2de03640 1 -- 172.21.15.107:0/584457956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc28107180 con 0x7fdc28106610 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.900+0000 7fdc2de03640 1 --2- 172.21.15.107:0/584457956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc28106a10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009820 tx=0x7fdc1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852beac3f7d67ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.900+0000 7fdc2ce01640 1 -- 172.21.15.107:0/584457956 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc18037070 con 0x7fdc28106610 2024-09-21T11:48:11.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc2ce01640 1 -- 172.21.15.107:0/584457956 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fdc18032870 con 0x7fdc28106610 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc2ce01640 1 -- 172.21.15.107:0/584457956 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc180305d0 con 0x7fdc28106610 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc30067640 1 -- 172.21.15.107:0/584457956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 msgr2=0x7fdc28106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc30067640 1 --2- 172.21.15.107:0/584457956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc28106a10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdc18009820 tx=0x7fdc1802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc30067640 1 -- 172.21.15.107:0/584457956 shutdown_connections 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc30067640 1 --2- 172.21.15.107:0/584457956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc28106a10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.901+0000 7fdc30067640 1 -- 172.21.15.107:0/584457956 >> 172.21.15.107:0/584457956 conn(0x7fdc28101f00 msgr2=0x7fdc28104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc30067640 1 -- 172.21.15.107:0/584457956 shutdown_connections 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc30067640 1 -- 172.21.15.107:0/584457956 wait complete. 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc30067640 1 Processor -- start 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc30067640 1 -- start start 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc30067640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc2807c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc30067640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2810aa90 con 0x7fdc28106610 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc2de03640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc2807c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc2de03640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc2807c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48754/0 (socket says 172.21.15.107:48754) 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.902+0000 7fdc2de03640 1 -- 172.21.15.107:0/1638695086 learned_addr learned my addr 172.21.15.107:0/1638695086 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc2de03640 1 -- 172.21.15.107:0/1638695086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2807c6e0 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc2de03640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc2807c180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc180097f0 tx=0x7fdc18030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc1803f020 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fdc18030e40 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2807aad0 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc18037040 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.903+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2807b050 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.904+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 3) ==== 51725+0+0 (secure 0 0 0) 0x7fdc18042020 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.904+0000 7fdc1effd640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fdc1403f690 0x7fdc14041b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.904+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc18074210 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.904+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2810ac10 con 0x7fdc28106610 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.904+0000 7fdc2d602640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fdc1403f690 0x7fdc14041b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.905+0000 7fdc2d602640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fdc1403f690 0x7fdc14041b50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098c0 tx=0x7fdc24006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:10.909+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc18049020 con 0x7fdc28106610 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.152+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdc2807b5b0 con 0x7fdc28106610 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.152+0000 7fdc1effd640 1 -- 172.21.15.107:0/1638695086 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fdc1804bd10 con 0x7fdc28106610 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fdc1403f690 msgr2=0x7fdc14041b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fdc1403f690 0x7fdc14041b50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc240098c0 tx=0x7fdc24006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 msgr2=0x7fdc2807c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc2807c180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc180097f0 tx=0x7fdc18030b00 comp rx=0 tx=0).stop 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 shutdown_connections 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fdc1403f690 0x7fdc14041b50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 --2- 172.21.15.107:0/1638695086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28106610 0x7fdc2807c180 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 >> 172.21.15.107:0/1638695086 conn(0x7fdc28101f00 msgr2=0x7fdc28102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 shutdown_connections 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:11.154+0000 7fdc30067640 1 -- 172.21.15.107:0/1638695086 wait complete. 2024-09-21T11:48:11.733 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-09-21T11:48:12.326 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:12 smithi107 ceph-mon[30293]: mgrmap e4: y(active, since 2s) 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T11:48:12.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.135+0000 7f986b87b640 1 Processor -- start 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.135+0000 7f986b87b640 1 -- start start 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.135+0000 7f986b87b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.135+0000 7f986b87b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9864079e50 con 0x7f9864079a50 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.136+0000 7f9869617640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.136+0000 7f9869617640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986410b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48770/0 (socket says 172.21.15.107:48770) 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.136+0000 7f9869617640 1 -- 172.21.15.107:0/1276841294 learned_addr learned my addr 172.21.15.107:0/1276841294 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.136+0000 7f9869617640 1 -- 172.21.15.107:0/1276841294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986410bb20 con 0x7f9864079a50 2024-09-21T11:48:12.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f9869617640 1 --2- 172.21.15.107:0/1276841294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986410b5e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9854009820 tx=0x7f985402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339c0280c0617487 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f985bfff640 1 -- 172.21.15.107:0/1276841294 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9854037070 con 0x7f9864079a50 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f985bfff640 1 -- 172.21.15.107:0/1276841294 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f9854032870 con 0x7f9864079a50 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f985bfff640 1 -- 172.21.15.107:0/1276841294 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98540305d0 con 0x7f9864079a50 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 -- 172.21.15.107:0/1276841294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 msgr2=0x7f986410b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 --2- 172.21.15.107:0/1276841294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986410b5e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9854009820 tx=0x7f985402a270 comp rx=0 tx=0).stop 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 -- 172.21.15.107:0/1276841294 shutdown_connections 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 --2- 172.21.15.107:0/1276841294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986410b5e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 -- 172.21.15.107:0/1276841294 >> 172.21.15.107:0/1276841294 conn(0x7f9864100df0 msgr2=0x7f9864103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 -- 172.21.15.107:0/1276841294 shutdown_connections 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.137+0000 7f986b87b640 1 -- 172.21.15.107:0/1276841294 wait complete. 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f986b87b640 1 Processor -- start 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f986b87b640 1 -- start start 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f986b87b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:12.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f986b87b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f986410cae0 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f9869617640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f9869617640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48774/0 (socket says 172.21.15.107:48774) 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f9869617640 1 -- 172.21.15.107:0/1168972525 learned_addr learned my addr 172.21.15.107:0/1168972525 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.138+0000 7f9869617640 1 -- 172.21.15.107:0/1168972525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986419cdf0 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.139+0000 7f9869617640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986419c890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98540097f0 tx=0x7f9854030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.139+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f985403f020 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.139+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f9854030dc0 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.139+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986419d080 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.139+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9854037040 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.139+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986419da40 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.140+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 51831+0+0 (secure 0 0 0) 0x7f9854042020 con 0x7f9864079a50 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.140+0000 7f985a7fc640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f985003f7f0 0x7f9850041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:12.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.140+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98540743f0 con 0x7f9864079a50 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.140+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986410b660 con 0x7f9864079a50 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.140+0000 7f9868e16640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f985003f7f0 0x7f9850041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.140+0000 7f9868e16640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f985003f7f0 0x7f9850041cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98600098c0 tx=0x7f9860006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.145+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9854049020 con 0x7f9864079a50 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.324+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f986419e140 con 0x7f9864079a50 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.325+0000 7f985a7fc640 1 -- 172.21.15.107:0/1168972525 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+376 (secure 0 0 0) 0x7f9854044df0 con 0x7f9864079a50 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f985003f7f0 msgr2=0x7f9850041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f985003f7f0 0x7f9850041cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98600098c0 tx=0x7f9860006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 msgr2=0x7f986419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986419c890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98540097f0 tx=0x7f9854030b00 comp rx=0 tx=0).stop 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 shutdown_connections 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f985003f7f0 0x7f9850041cb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 --2- 172.21.15.107:0/1168972525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9864079a50 0x7f986419c890 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:12.924 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 >> 172.21.15.107:0/1168972525 conn(0x7f9864100df0 msgr2=0x7f9864101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 shutdown_connections 2024-09-21T11:48:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:12.327+0000 7f986b87b640 1 -- 172.21.15.107:0/1168972525 wait complete. 2024-09-21T11:48:12.925 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-09-21T11:48:13.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1168972525' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T11:48:14.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1741914911' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T11:48:14.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:14 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ignoring --setuser ceph since I am not root 2024-09-21T11:48:14.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:14 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ignoring --setgroup ceph since I am not root 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80ff1d640 1 Processor -- start 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80ff1d640 1 -- start start 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80ff1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd808105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80ff1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd808105f00 con 0x7fd808105510 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80dcb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd808105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80dcb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd808105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48784/0 (socket says 172.21.15.107:48784) 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.333+0000 7fd80dcb9640 1 -- 172.21.15.107:0/2766561646 learned_addr learned my addr 172.21.15.107:0/2766561646 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.334+0000 7fd80dcb9640 1 -- 172.21.15.107:0/2766561646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd808106080 con 0x7fd808105510 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.334+0000 7fd80dcb9640 1 --2- 172.21.15.107:0/2766561646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd808105910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009820 tx=0x7fd7f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0d2ff6d95cfc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.334+0000 7fd80ccb7640 1 -- 172.21.15.107:0/2766561646 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7f8037070 con 0x7fd808105510 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.334+0000 7fd80ccb7640 1 -- 172.21.15.107:0/2766561646 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fd7f8032870 con 0x7fd808105510 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.334+0000 7fd80ccb7640 1 -- 172.21.15.107:0/2766561646 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7f80305d0 con 0x7fd808105510 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 -- 172.21.15.107:0/2766561646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 msgr2=0x7fd808105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 --2- 172.21.15.107:0/2766561646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd808105910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009820 tx=0x7fd7f802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 -- 172.21.15.107:0/2766561646 shutdown_connections 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 --2- 172.21.15.107:0/2766561646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd808105910 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 -- 172.21.15.107:0/2766561646 >> 172.21.15.107:0/2766561646 conn(0x7fd808100e00 msgr2=0x7fd808103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 -- 172.21.15.107:0/2766561646 shutdown_connections 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.335+0000 7fd80ff1d640 1 -- 172.21.15.107:0/2766561646 wait complete. 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80ff1d640 1 Processor -- start 2024-09-21T11:48:14.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80ff1d640 1 -- start start 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80ff1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd80807a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80ff1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd808109990 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80dcb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd80807a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80dcb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd80807a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48792/0 (socket says 172.21.15.107:48792) 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80dcb9640 1 -- 172.21.15.107:0/1741914911 learned_addr learned my addr 172.21.15.107:0/1741914911 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80dcb9640 1 -- 172.21.15.107:0/1741914911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80807a5c0 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd80dcb9640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd80807a060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80090b0 tx=0x7fd7f80307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7f803f020 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fd7f8030e20 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.336+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7f8037040 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.337+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80819d360 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.337+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80819d830 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.337+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 51831+0+0 (secure 0 0 0) 0x7fd7f8042020 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.338+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd808109b10 con 0x7fd808105510 2024-09-21T11:48:14.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.338+0000 7fd7feffd640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fd7f403f7f0 0x7fd7f4041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.338+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd7f80743d0 con 0x7fd808105510 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.338+0000 7fd80d4b8640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fd7f403f7f0 0x7fd7f4041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.338+0000 7fd80d4b8640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fd7f403f7f0 0x7fd7f4041cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8040098c0 tx=0x7fd804006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.342+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd7f8049020 con 0x7fd808105510 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:13.570+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd80819dba0 con 0x7fd808105510 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.216+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd7f803f1c0 con 0x7fd808105510 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.218+0000 7fd7feffd640 1 -- 172.21.15.107:0/1741914911 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 5) ==== 51842+0+0 (secure 0 0 0) 0x7fd7f8049cf0 con 0x7fd808105510 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fd7f403f7f0 msgr2=0x7fd7f4041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fd7f403f7f0 0x7fd7f4041cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8040098c0 tx=0x7fd804006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 msgr2=0x7fd80807a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd80807a060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80090b0 tx=0x7fd7f80307c0 comp rx=0 tx=0).stop 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 shutdown_connections 2024-09-21T11:48:14.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7fd7f403f7f0 0x7fd7f4041cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:14.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 --2- 172.21.15.107:0/1741914911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808105510 0x7fd80807a060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:14.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 >> 172.21.15.107:0/1741914911 conn(0x7fd808100e00 msgr2=0x7fd8081019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:14.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 shutdown_connections 2024-09-21T11:48:14.887 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:14.220+0000 7fd80ff1d640 1 -- 172.21.15.107:0/1741914911 wait complete. 2024-09-21T11:48:15.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1741914911' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T11:48:15.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:15 smithi107 ceph-mon[30293]: mgrmap e5: y(active, since 5s) 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.347+0000 7f1960057640 1 Processor -- start 2024-09-21T11:48:16.115 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f1960057640 1 -- start start 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f1960057640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f19581058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f1960057640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1958105ed0 con 0x7f19581054e0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f195ddf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f19581058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f195ddf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f19581058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47466/0 (socket says 172.21.15.107:47466) 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f195ddf3640 1 -- 172.21.15.107:0/382719088 learned_addr learned my addr 172.21.15.107:0/382719088 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.348+0000 7f195ddf3640 1 -- 172.21.15.107:0/382719088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958106050 con 0x7f19581054e0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.349+0000 7f195ddf3640 1 --2- 172.21.15.107:0/382719088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f19581058e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1954009820 tx=0x7f195402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94df2c093c40c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.349+0000 7f195cdf1640 1 -- 172.21.15.107:0/382719088 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954037070 con 0x7f19581054e0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.349+0000 7f195cdf1640 1 -- 172.21.15.107:0/382719088 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f1954032870 con 0x7f19581054e0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.349+0000 7f195cdf1640 1 -- 172.21.15.107:0/382719088 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19540305d0 con 0x7f19581054e0 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 -- 172.21.15.107:0/382719088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 msgr2=0x7f19581058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 --2- 172.21.15.107:0/382719088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f19581058e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1954009820 tx=0x7f195402a270 comp rx=0 tx=0).stop 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 -- 172.21.15.107:0/382719088 shutdown_connections 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 --2- 172.21.15.107:0/382719088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f19581058e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:16.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 -- 172.21.15.107:0/382719088 >> 172.21.15.107:0/382719088 conn(0x7f1958100db0 msgr2=0x7f19581031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 -- 172.21.15.107:0/382719088 shutdown_connections 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.350+0000 7f1960057640 1 -- 172.21.15.107:0/382719088 wait complete. 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f1960057640 1 Processor -- start 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f1960057640 1 -- start start 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f1960057640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f1958190400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f1960057640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1958109960 con 0x7f19581054e0 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f195ddf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f1958190400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f195ddf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f1958190400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47470/0 (socket says 172.21.15.107:47470) 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f195ddf3640 1 -- 172.21.15.107:0/2191298045 learned_addr learned my addr 172.21.15.107:0/2191298045 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f195ddf3640 1 -- 172.21.15.107:0/2191298045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958190960 con 0x7f19581054e0 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f195ddf3640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f1958190400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f19540090b0 tx=0x7f1954030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.351+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f195403f020 con 0x7f19581054e0 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.352+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f1954030e10 con 0x7f19581054e0 2024-09-21T11:48:16.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.352+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195818ed50 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.352+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1954037040 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.352+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195818f2d0 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.352+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 51842+0+0 (secure 0 0 0) 0x7f1954042020 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.353+0000 7f194effd640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 0x7f1944041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.353+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958109ae0 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.353+0000 7f195d5f2640 1 -- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 msgr2=0x7f1944041d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1943709907 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.353+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1954074420 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.353+0000 7f195d5f2640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 0x7f1944041d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.357+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1954049020 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.554+0000 7f195d5f2640 1 -- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 msgr2=0x7f1944041d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1943709907 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.554+0000 7f195d5f2640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 0x7f1944041d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.585+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f195818f7e0 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.586+0000 7f194effd640 1 -- 172.21.15.107:0/2191298045 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1954074180 con 0x7f19581054e0 2024-09-21T11:48:16.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 msgr2=0x7f1944041d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 0x7f1944041d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 msgr2=0x7f1958190400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f1958190400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f19540090b0 tx=0x7f1954030b50 comp rx=0 tx=0).stop 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 shutdown_connections 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f194403f840 0x7f1944041d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 --2- 172.21.15.107:0/2191298045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19581054e0 0x7f1958190400 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 >> 172.21.15.107:0/2191298045 conn(0x7f1958100db0 msgr2=0x7f1958079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 shutdown_connections 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:15.588+0000 7f1960057640 1 -- 172.21.15.107:0/2191298045 wait complete. 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-09-21T11:48:16.119 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-09-21T11:48:16.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2191298045' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: Active manager daemon y restarted 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: Activating manager daemon y 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: mgrmap e6: y(active, starting, since 0.00346164s) 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:48:19.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:19 smithi107 ceph-mon[30293]: Manager daemon y is now available 2024-09-21T11:48:20.855 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:20.855 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T11:48:20.855 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T11:48:20.855 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:20.855 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f7a00963640 1 Processor -- start 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f7a00963640 1 -- start start 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f7a00963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f7a00963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1096c0 con 0x7f79fc079bc0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f79fa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f79fa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47472/0 (socket says 172.21.15.107:47472) 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.581+0000 7f79fa59c640 1 -- 172.21.15.107:0/566762328 learned_addr learned my addr 172.21.15.107:0/566762328 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.582+0000 7f79fa59c640 1 -- 172.21.15.107:0/566762328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc109840 con 0x7f79fc079bc0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.582+0000 7f79fa59c640 1 --2- 172.21.15.107:0/566762328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc109180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009820 tx=0x7f79ec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2b4b78d6258b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f79f959a640 1 -- 172.21.15.107:0/566762328 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79ec037070 con 0x7f79fc079bc0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f79f959a640 1 -- 172.21.15.107:0/566762328 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f79ec032870 con 0x7f79fc079bc0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f79f959a640 1 -- 172.21.15.107:0/566762328 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79ec030660 con 0x7f79fc079bc0 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 -- 172.21.15.107:0/566762328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 msgr2=0x7f79fc109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:20.856 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 --2- 172.21.15.107:0/566762328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc109180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009820 tx=0x7f79ec02a300 comp rx=0 tx=0).stop 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 -- 172.21.15.107:0/566762328 shutdown_connections 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 --2- 172.21.15.107:0/566762328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc109180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 -- 172.21.15.107:0/566762328 >> 172.21.15.107:0/566762328 conn(0x7f79fc100df0 msgr2=0x7f79fc103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 -- 172.21.15.107:0/566762328 shutdown_connections 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.583+0000 7f7a00963640 1 -- 172.21.15.107:0/566762328 wait complete. 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.584+0000 7f7a00963640 1 Processor -- start 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.584+0000 7f7a00963640 1 -- start start 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.584+0000 7f7a00963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc1067d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.584+0000 7f7a00963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10a8b0 con 0x7f79fc079bc0 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79fa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc1067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79fa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc1067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47486/0 (socket says 172.21.15.107:47486) 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79fa59c640 1 -- 172.21.15.107:0/4177363489 learned_addr learned my addr 172.21.15.107:0/4177363489 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79fa59c640 1 -- 172.21.15.107:0/4177363489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc106d30 con 0x7f79fc079bc0 2024-09-21T11:48:20.857 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79fa59c640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc1067d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000c00 tx=0x7f79ec030a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79ec037040 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f79ec040070 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc105120 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79ec03a390 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.585+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc1056a0 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.586+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 51842+0+0 (secure 0 0 0) 0x7f79ec03a610 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.587+0000 7f79eb7fe640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.587+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 --> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f79fc106fc0 con 0x7f79e403f7f0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.587+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f79ec074220 con 0x7f79fc079bc0 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.587+0000 7f79f9d9b640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 msgr2=0x7f79e4041cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1943709907 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.587+0000 7f79f9d9b640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.788+0000 7f79f9d9b640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 msgr2=0x7f79e4041cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1943709907 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:16.788+0000 7f79f9d9b640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T11:48:20.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:17.189+0000 7f79f9d9b640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 msgr2=0x7f79e4041cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1943709907 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:17.189+0000 7f79f9d9b640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:17.990+0000 7f79f9d9b640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 msgr2=0x7f79e4041cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1943709907 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:17.990+0000 7f79f9d9b640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:19.248+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) ==== 51609+0+0 (secure 0 0 0) 0x7f79ec073020 con 0x7f79fc079bc0 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:19.248+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 msgr2=0x7f79e4041cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:19.248+0000 7f79eb7fe640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.249+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) ==== 51736+0+0 (secure 0 0 0) 0x7f79ec0457e0 con 0x7f79fc079bc0 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.249+0000 7f79eb7fe640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f79e4042f10 0x7f79e4045300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.249+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f79ec048e50 con 0x7f79e4042f10 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.249+0000 7f79f9d9b640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f79e4042f10 0x7f79e4045300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.250+0000 7f79f9d9b640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f79e4042f10 0x7f79e4045300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f79f0003980 tx=0x7f79f0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.250+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f79ec048e50 con 0x7f79e4042f10 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.257+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f79fc1071f0 con 0x7f79e4042f10 2024-09-21T11:48:20.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f79eb7fe640 1 -- 172.21.15.107:0/4177363489 <== mgr.14118 v2:172.21.15.107:6800/1807990149 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f79fc1071f0 con 0x7f79e4042f10 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f79e4042f10 msgr2=0x7f79e4045300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f79e4042f10 0x7f79e4045300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f79f0003980 tx=0x7f79f0006e10 comp rx=0 tx=0).stop 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 msgr2=0x7f79fc1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc1067d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000c00 tx=0x7f79ec030a60 comp rx=0 tx=0).stop 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 shutdown_connections 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f79e4042f10 0x7f79e4045300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:6800/1943709907,v1:172.21.15.107:6801/1943709907] conn(0x7f79e403f7f0 0x7f79e4041cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 --2- 172.21.15.107:0/4177363489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79fc079bc0 0x7f79fc1067d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 >> 172.21.15.107:0/4177363489 conn(0x7f79fc100df0 msgr2=0x7f79fc101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 shutdown_connections 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:20.258+0000 7f7a00963640 1 -- 172.21.15.107:0/4177363489 wait complete. 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-09-21T11:48:20.860 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying orchestrator module is enabled... 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: mgrmap e7: y(active, since 1.00526s) 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:48:21.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:48:21.397+0000 7fb028ceb640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:11:48:21] ENGINE Error in HTTPServer.serve 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: Traceback (most recent call last): 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._connections.run(self.expiration_interval) 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._run(expiration_interval) 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: s = self.context.wrap_socket( 2024-09-21T11:48:21.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: return self.sslsocket_class._create( 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self.do_handshake() 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._sslobj.do_handshake() 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:48:21.506 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:21 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:48:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:22 smithi107 ceph-mon[30293]: Found migration_current of "None". Setting to last migration. 2024-09-21T11:48:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:22 smithi107 ceph-mon[30293]: mgrmap e8: y(active, since 2s) 2024-09-21T11:48:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:22 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/184350276' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff74e235640 1 Processor -- start 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff74e235640 1 -- start start 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff74e235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff7481058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff74e235640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff748105eb0 con 0x7ff7481054c0 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff7481058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff7481058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47540/0 (socket says 172.21.15.107:47540) 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff7477fe640 1 -- 172.21.15.107:0/1406369763 learned_addr learned my addr 172.21.15.107:0/1406369763 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:22.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.278+0000 7ff7477fe640 1 -- 172.21.15.107:0/1406369763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff748106030 con 0x7ff7481054c0 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.279+0000 7ff7477fe640 1 --2- 172.21.15.107:0/1406369763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff7481058c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff738009820 tx=0x7ff73802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bf8ef223883345 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.279+0000 7ff7467fc640 1 -- 172.21.15.107:0/1406369763 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff738037070 con 0x7ff7481054c0 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff7467fc640 1 -- 172.21.15.107:0/1406369763 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7ff738032870 con 0x7ff7481054c0 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff7467fc640 1 -- 172.21.15.107:0/1406369763 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7380305d0 con 0x7ff7481054c0 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 -- 172.21.15.107:0/1406369763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 msgr2=0x7ff7481058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 --2- 172.21.15.107:0/1406369763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff7481058c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff738009820 tx=0x7ff73802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 -- 172.21.15.107:0/1406369763 shutdown_connections 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 --2- 172.21.15.107:0/1406369763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff7481058c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 -- 172.21.15.107:0/1406369763 >> 172.21.15.107:0/1406369763 conn(0x7ff748100df0 msgr2=0x7ff748103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 -- 172.21.15.107:0/1406369763 shutdown_connections 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.280+0000 7ff74e235640 1 -- 172.21.15.107:0/1406369763 wait complete. 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff74e235640 1 Processor -- start 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff74e235640 1 -- start start 2024-09-21T11:48:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff74e235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff74819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff74e235640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff748109940 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff74819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff74819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47554/0 (socket says 172.21.15.107:47554) 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.281+0000 7ff7477fe640 1 -- 172.21.15.107:0/184350276 learned_addr learned my addr 172.21.15.107:0/184350276 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff7477fe640 1 -- 172.21.15.107:0/184350276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74819ac50 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff7477fe640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff74819a6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7380097f0 tx=0x7ff738030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff73803f020 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff74e235640 1 -- 172.21.15.107:0/184350276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74819aee0 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7ff738030dc0 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff738037040 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.282+0000 7ff74e235640 1 -- 172.21.15.107:0/184350276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74819b8a0 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.283+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff748109ac0 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.284+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 7) ==== 51736+0+0 (secure 0 0 0) 0x7ff738042020 con 0x7ff7481054c0 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.284+0000 7ff744ff9640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7ff73003f6e0 0x7ff730041ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:22.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.284+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff738074690 con 0x7ff7481054c0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.284+0000 7ff746ffd640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7ff73003f6e0 0x7ff730041ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.285+0000 7ff746ffd640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7ff73003f6e0 0x7ff730041ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098c0 tx=0x7ff73c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.288+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff73803b940 con 0x7ff7481054c0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.369+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 8) ==== 51842+0+0 (secure 0 0 0) 0x7ff73804c360 con 0x7ff7481054c0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:21.517+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "orchestrator"} v 0) -- 0x7ff748105940 con 0x7ff7481054c0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.369+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "orchestrator"}]=0 module 'orchestrator' is already enabled (always-on) v9) ==== 143+0+0 (secure 0 0 0) 0x7ff73803a6d0 con 0x7ff7481054c0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.371+0000 7ff744ff9640 1 -- 172.21.15.107:0/184350276 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7ff73803fc50 con 0x7ff7481054c0 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.372+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7ff73003f6e0 msgr2=0x7ff730041ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.372+0000 7ff72e7fc640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7ff73003f6e0 0x7ff730041ba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0098c0 tx=0x7ff73c006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.372+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 msgr2=0x7ff74819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.372+0000 7ff72e7fc640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff74819a6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7380097f0 tx=0x7ff738030b00 comp rx=0 tx=0).stop 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.373+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 shutdown_connections 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.373+0000 7ff72e7fc640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7ff73003f6e0 0x7ff730041ba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:22.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.373+0000 7ff72e7fc640 1 --2- 172.21.15.107:0/184350276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7481054c0 0x7ff74819a6f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:22.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.373+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 >> 172.21.15.107:0/184350276 conn(0x7ff748100df0 msgr2=0x7ff748101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:22.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.373+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 shutdown_connections 2024-09-21T11:48:22.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:22.373+0000 7ff72e7fc640 1 -- 172.21.15.107:0/184350276 wait complete. 2024-09-21T11:48:22.965 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:21] ENGINE Bus STARTING 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:21] ENGINE Serving on https://172.21.15.107:7150 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:21] ENGINE Error in HTTPServer.serve 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:48:23.356 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:48:23.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:21] ENGINE Serving on http://172.21.15.107:8765 2024-09-21T11:48:23.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:21] ENGINE Bus STARTED 2024-09-21T11:48:23.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/184350276' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T11:48:23.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:23 smithi107 ceph-mon[30293]: mgrmap e9: y(active, since 3s) 2024-09-21T11:48:24.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.438+0000 7f5e1264f640 1 Processor -- start 2024-09-21T11:48:24.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.438+0000 7f5e1264f640 1 -- start start 2024-09-21T11:48:24.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.438+0000 7f5e1264f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:24.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.438+0000 7f5e1264f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c07c110 con 0x7f5e0c07bc60 2024-09-21T11:48:24.212 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.438+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.439+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47558/0 (socket says 172.21.15.107:47558) 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.439+0000 7f5e0bfff640 1 -- 172.21.15.107:0/790093683 learned_addr learned my addr 172.21.15.107:0/790093683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.439+0000 7f5e0bfff640 1 -- 172.21.15.107:0/790093683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c07a800 con 0x7f5e0c07bc60 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.439+0000 7f5e0bfff640 1 --2- 172.21.15.107:0/790093683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c07a2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009820 tx=0x7f5dfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6c62cf57e6147f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e0affd640 1 -- 172.21.15.107:0/790093683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc037070 con 0x7f5e0c07bc60 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e0affd640 1 -- 172.21.15.107:0/790093683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f5dfc032870 con 0x7f5e0c07bc60 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e0affd640 1 -- 172.21.15.107:0/790093683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc0305d0 con 0x7f5e0c07bc60 2024-09-21T11:48:24.213 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 -- 172.21.15.107:0/790093683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 msgr2=0x7f5e0c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 --2- 172.21.15.107:0/790093683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c07a2c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009820 tx=0x7f5dfc02a270 comp rx=0 tx=0).stop 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 -- 172.21.15.107:0/790093683 shutdown_connections 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 --2- 172.21.15.107:0/790093683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c07a2c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 -- 172.21.15.107:0/790093683 >> 172.21.15.107:0/790093683 conn(0x7f5e0c101f30 msgr2=0x7f5e0c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 -- 172.21.15.107:0/790093683 shutdown_connections 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.440+0000 7f5e1264f640 1 -- 172.21.15.107:0/790093683 wait complete. 2024-09-21T11:48:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e1264f640 1 Processor -- start 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e1264f640 1 -- start start 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e1264f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e1264f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10cab0 con 0x7f5e0c07bc60 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47562/0 (socket says 172.21.15.107:47562) 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.441+0000 7f5e0bfff640 1 -- 172.21.15.107:0/2950306485 learned_addr learned my addr 172.21.15.107:0/2950306485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e0bfff640 1 -- 172.21.15.107:0/2950306485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c19cd90 con 0x7f5e0c07bc60 2024-09-21T11:48:24.215 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e0bfff640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c19c830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0097f0 tx=0x7f5dfc030b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc03f020 con 0x7f5e0c07bc60 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f5dfc030dc0 con 0x7f5e0c07bc60 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c19d020 con 0x7f5e0c07bc60 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dfc037040 con 0x7f5e0c07bc60 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.442+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c19d9e0 con 0x7f5e0c07bc60 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.443+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f5dfc042020 con 0x7f5e0c07bc60 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.443+0000 7f5e097fa640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5df403f7f0 0x7f5df4041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:24.216 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.443+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5dfc074760 con 0x7f5e0c07bc60 2024-09-21T11:48:24.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.443+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c07a340 con 0x7f5e0c07bc60 2024-09-21T11:48:24.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.443+0000 7f5e0b7fe640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5df403f7f0 0x7f5df4041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:24.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.443+0000 7f5e0b7fe640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5df403f7f0 0x7f5df4041cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e000098c0 tx=0x7f5e00006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:24.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.448+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5dfc049020 con 0x7f5e0c07bc60 2024-09-21T11:48:24.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.639+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f5e0c103260 con 0x7f5df403f7f0 2024-09-21T11:48:24.217 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.647+0000 7f5e097fa640 1 -- 172.21.15.107:0/2950306485 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e0c103260 con 0x7f5df403f7f0 2024-09-21T11:48:24.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5df403f7f0 msgr2=0x7f5df4041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:24.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5df403f7f0 0x7f5df4041cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e000098c0 tx=0x7f5e00006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:24.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 msgr2=0x7f5e0c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:24.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c19c830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0097f0 tx=0x7f5dfc030b00 comp rx=0 tx=0).stop 2024-09-21T11:48:24.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 shutdown_connections 2024-09-21T11:48:24.218 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5df403f7f0 0x7f5df4041cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:24.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 --2- 172.21.15.107:0/2950306485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c07bc60 0x7f5e0c19c830 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:24.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.649+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 >> 172.21.15.107:0/2950306485 conn(0x7f5e0c101f30 msgr2=0x7f5e0c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:24.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.650+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 shutdown_connections 2024-09-21T11:48:24.219 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:23.650+0000 7f5e1264f640 1 -- 172.21.15.107:0/2950306485 wait complete. 2024-09-21T11:48:24.845 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:24 smithi107 ceph-mon[30293]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:24.845 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:24 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:24.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:24 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:25.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7eacd3640 1 Processor -- start 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7eacd3640 1 -- start start 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7eacd3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e4106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7eacd3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4107000 con 0x7fe7e4106610 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7e8a6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e4106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7e8a6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e4106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60300/0 (socket says 172.21.15.107:60300) 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7e8a6f640 1 -- 172.21.15.107:0/2719585683 learned_addr learned my addr 172.21.15.107:0/2719585683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.654+0000 7fe7e8a6f640 1 -- 172.21.15.107:0/2719585683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e4107180 con 0x7fe7e4106610 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.655+0000 7fe7e8a6f640 1 --2- 172.21.15.107:0/2719585683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e4106a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009820 tx=0x7fe7d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3bc0dca6f9541f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.655+0000 7fe7e37fe640 1 -- 172.21.15.107:0/2719585683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d8037070 con 0x7fe7e4106610 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.655+0000 7fe7e37fe640 1 -- 172.21.15.107:0/2719585683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe7d8032870 con 0x7fe7e4106610 2024-09-21T11:48:25.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.655+0000 7fe7e37fe640 1 -- 172.21.15.107:0/2719585683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d80305d0 con 0x7fe7e4106610 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 -- 172.21.15.107:0/2719585683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 msgr2=0x7fe7e4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 --2- 172.21.15.107:0/2719585683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e4106a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009820 tx=0x7fe7d802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 -- 172.21.15.107:0/2719585683 shutdown_connections 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 --2- 172.21.15.107:0/2719585683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e4106a10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 -- 172.21.15.107:0/2719585683 >> 172.21.15.107:0/2719585683 conn(0x7fe7e4101f00 msgr2=0x7fe7e4104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 -- 172.21.15.107:0/2719585683 shutdown_connections 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 -- 172.21.15.107:0/2719585683 wait complete. 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.656+0000 7fe7eacd3640 1 Processor -- start 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7eacd3640 1 -- start start 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7eacd3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7eacd3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e410aa90 con 0x7fe7e4106610 2024-09-21T11:48:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e8a6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e8a6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60310/0 (socket says 172.21.15.107:60310) 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e8a6f640 1 -- 172.21.15.107:0/4141542389 learned_addr learned my addr 172.21.15.107:0/4141542389 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e8a6f640 1 -- 172.21.15.107:0/4141542389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419aba0 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e8a6f640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e419a640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80090b0 tx=0x7fe7d80307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d803f020 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fe7d8030e20 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419ae30 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.657+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7d8037040 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.658+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e419b7f0 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.658+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7fe7d8042020 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.659+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e410ac10 con 0x7fe7e4106610 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.659+0000 7fe7e1ffb640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fe7d403f7f0 0x7fe7d4041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:25.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.659+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7d8074820 con 0x7fe7e4106610 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.659+0000 7fe7e3fff640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fe7d403f7f0 0x7fe7d4041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.659+0000 7fe7e3fff640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fe7d403f7f0 0x7fe7d4041cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00098c0 tx=0x7fe7d0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.663+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7d8049020 con 0x7fe7e4106610 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.843+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe7e4103200 con 0x7fe7d403f7f0 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.844+0000 7fe7e1ffb640 1 -- 172.21.15.107:0/4141542389 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe7e4103200 con 0x7fe7d403f7f0 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fe7d403f7f0 msgr2=0x7fe7d4041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fe7d403f7f0 0x7fe7d4041cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00098c0 tx=0x7fe7d0006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 msgr2=0x7fe7e419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e419a640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80090b0 tx=0x7fe7d80307c0 comp rx=0 tx=0).stop 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 shutdown_connections 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fe7d403f7f0 0x7fe7d4041cb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 --2- 172.21.15.107:0/4141542389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7e4106610 0x7fe7e419a640 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 >> 172.21.15.107:0/4141542389 conn(0x7fe7e4101f00 msgr2=0x7fe7e4102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:25.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 shutdown_connections 2024-09-21T11:48:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:24.847+0000 7fe7eacd3640 1 -- 172.21.15.107:0/4141542389 wait complete. 2024-09-21T11:48:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-09-21T11:48:25.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:25 smithi107 ceph-mon[30293]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: Generating public/private rsa key pair. 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: Your identification has been saved in /tmp/tmprhtc5u_x/key 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: Your public key has been saved in /tmp/tmprhtc5u_x/key.pub 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: The key fingerprint is: 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: SHA256:7OkIBKA3M/e0+jYDgHHgOHSSjpW1wJRbU2CaUnqxB+8 ceph-1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: The key's randomart image is: 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: +---[RSA 3072]----+ 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: |o*B=+o. | 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: |*=OXo. | 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: |O*@o=.. | 2024-09-21T11:48:26.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: |o*oO o o | 2024-09-21T11:48:26.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: | .E o S | 2024-09-21T11:48:26.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: | ... . . | 2024-09-21T11:48:26.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: | o. o | 2024-09-21T11:48:26.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: | o+o | 2024-09-21T11:48:26.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: | .oo. | 2024-09-21T11:48:26.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: +----[SHA256]-----+ 2024-09-21T11:48:26.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-mon[30293]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:26.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-mon[30293]: Generating ssh key... 2024-09-21T11:48:26.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:26.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:26 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:26.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.917+0000 7faa0fac2640 1 Processor -- start 2024-09-21T11:48:26.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.917+0000 7faa0fac2640 1 -- start start 2024-09-21T11:48:26.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.917+0000 7faa0fac2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa081091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.917+0000 7faa0fac2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faa08109720 con 0x7faa08079b60 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.918+0000 7faa0d85e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa081091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.918+0000 7faa0d85e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa081091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60322/0 (socket says 172.21.15.107:60322) 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.918+0000 7faa0d85e640 1 -- 172.21.15.107:0/704080849 learned_addr learned my addr 172.21.15.107:0/704080849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.918+0000 7faa0d85e640 1 -- 172.21.15.107:0/704080849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa081098a0 con 0x7faa08079b60 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.918+0000 7faa0d85e640 1 --2- 172.21.15.107:0/704080849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa081091e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa04009820 tx=0x7faa0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ea17afb7ca5dba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0c85c640 1 -- 172.21.15.107:0/704080849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa04037070 con 0x7faa08079b60 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0c85c640 1 -- 172.21.15.107:0/704080849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7faa04032870 con 0x7faa08079b60 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0c85c640 1 -- 172.21.15.107:0/704080849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa040305d0 con 0x7faa08079b60 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 -- 172.21.15.107:0/704080849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 msgr2=0x7faa081091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 --2- 172.21.15.107:0/704080849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa081091e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa04009820 tx=0x7faa0402a270 comp rx=0 tx=0).stop 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 -- 172.21.15.107:0/704080849 shutdown_connections 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 --2- 172.21.15.107:0/704080849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa081091e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 -- 172.21.15.107:0/704080849 >> 172.21.15.107:0/704080849 conn(0x7faa08100df0 msgr2=0x7faa08103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 -- 172.21.15.107:0/704080849 shutdown_connections 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.919+0000 7faa0fac2640 1 -- 172.21.15.107:0/704080849 wait complete. 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0fac2640 1 Processor -- start 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0fac2640 1 -- start start 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0fac2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa0819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0fac2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faa0810a840 con 0x7faa08079b60 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0d85e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa0819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0d85e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa0819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60332/0 (socket says 172.21.15.107:60332) 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.920+0000 7faa0d85e640 1 -- 172.21.15.107:0/2012467809 learned_addr learned my addr 172.21.15.107:0/2012467809 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7faa0d85e640 1 -- 172.21.15.107:0/2012467809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0819ab80 con 0x7faa08079b60 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7faa0d85e640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa0819a620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa040090b0 tx=0x7faa04030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa0403f020 con 0x7faa08079b60 2024-09-21T11:48:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7faa04030e10 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa04037040 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0819ae10 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.921+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0819b850 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.922+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7faa04042020 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.922+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa08107250 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.922+0000 7fa9feffd640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fa9f403f840 0x7fa9f4041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.922+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa040746e0 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.922+0000 7faa0d05d640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fa9f403f840 0x7fa9f4041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.923+0000 7faa0d05d640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fa9f403f840 0x7fa9f4041d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80098c0 tx=0x7fa9f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:25.927+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa04049020 con 0x7faa08079b60 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.109+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7faa0819c030 con 0x7fa9f403f840 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.340+0000 7fa9feffd640 1 -- 172.21.15.107:0/2012467809 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa0819c030 con 0x7fa9f403f840 2024-09-21T11:48:26.906 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fa9f403f840 msgr2=0x7fa9f4041d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fa9f403f840 0x7fa9f4041d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80098c0 tx=0x7fa9f8006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 msgr2=0x7faa0819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa0819a620 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa040090b0 tx=0x7faa04030b50 comp rx=0 tx=0).stop 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 shutdown_connections 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fa9f403f840 0x7fa9f4041d00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 --2- 172.21.15.107:0/2012467809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa08079b60 0x7faa0819a620 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 >> 172.21.15.107:0/2012467809 conn(0x7faa08100df0 msgr2=0x7faa08101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 shutdown_connections 2024-09-21T11:48:26.907 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:26.343+0000 7faa0fac2640 1 -- 172.21.15.107:0/2012467809 wait complete. 2024-09-21T11:48:28.077 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmi6N2/z/7fw83ritsvqKgbLCCyfCu+B+dlD2ZTYj9sIsBR3gqPQ6GHBguZNl0YORpmlI86IPqVgBV9mX14cQypwfTE8IY6P3VMKDBVyc8Gvrol5y0h1WKz8xWST9r/gqkfIO64jBP0pBNjQktL/NBgIGfohyw2satdthCDYdU7ut2L5+8p5Uq8E+W+GoeKxXOTLS76XT01eNIKejdiDM9EWRWErA2yzMz1QhteE1IIp4rv+TEJH87ydTiuvXPm3jYqIpf6Wd3u/eSeOO4E349NJhK9RNVdOvVokxWmgDXdNibmjARbPweFG74QblCjcmtREmfZKTKTTvDvr5El5qhFgH/CcG7e0DxdfyNuHb3bGRghGu7cOF+x034rUWzZYJcSv1YDNkWuzgmgo5TpcILnFSWbHNz6z9cyGKUeSo5I/zfl9gQkgTKtbpEnFx+vXjJ3h084XtGdzK/7sc7HOUR8XOO7NWxoEaVkuVolz6bmCBXmSN+Ag7Qvl1zCMl6N08= ceph-1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f32351d3640 1 Processor -- start 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f32351d3640 1 -- start start 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f32351d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f3230106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f32351d3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3230107000 con 0x7f3230106610 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f322ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f3230106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f322ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f3230106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60342/0 (socket says 172.21.15.107:60342) 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f322ed9d640 1 -- 172.21.15.107:0/352682070 learned_addr learned my addr 172.21.15.107:0/352682070 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.317+0000 7f322ed9d640 1 -- 172.21.15.107:0/352682070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3230107180 con 0x7f3230106610 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.318+0000 7f322ed9d640 1 --2- 172.21.15.107:0/352682070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f3230106a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3220009820 tx=0x7f322002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d730b90422c5f460 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.318+0000 7f322dd9b640 1 -- 172.21.15.107:0/352682070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3220037070 con 0x7f3230106610 2024-09-21T11:48:28.078 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.318+0000 7f322dd9b640 1 -- 172.21.15.107:0/352682070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f3220032870 con 0x7f3230106610 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.318+0000 7f322dd9b640 1 -- 172.21.15.107:0/352682070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32200305d0 con 0x7f3230106610 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 -- 172.21.15.107:0/352682070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 msgr2=0x7f3230106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 --2- 172.21.15.107:0/352682070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f3230106a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3220009820 tx=0x7f322002a270 comp rx=0 tx=0).stop 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 -- 172.21.15.107:0/352682070 shutdown_connections 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 --2- 172.21.15.107:0/352682070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f3230106a10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 -- 172.21.15.107:0/352682070 >> 172.21.15.107:0/352682070 conn(0x7f3230101f00 msgr2=0x7f3230104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 -- 172.21.15.107:0/352682070 shutdown_connections 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 -- 172.21.15.107:0/352682070 wait complete. 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.319+0000 7f32351d3640 1 Processor -- start 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f32351d3640 1 -- start start 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f32351d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f323007c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f32351d3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f323010aa90 con 0x7f3230106610 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f322ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f323007c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f322ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f323007c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60356/0 (socket says 172.21.15.107:60356) 2024-09-21T11:48:28.079 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f322ed9d640 1 -- 172.21.15.107:0/1242766059 learned_addr learned my addr 172.21.15.107:0/1242766059 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f322ed9d640 1 -- 172.21.15.107:0/1242766059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323007c6e0 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f322ed9d640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f323007c180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32200090b0 tx=0x7f32200307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f322003f020 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.320+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f3220030e20 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.321+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323007aad0 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.321+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3220037040 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.321+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323007b050 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.322+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323010ac10 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.322+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f3220042020 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.322+0000 7f3217fff640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f321803f840 0x7f3218041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.322+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f32200748a0 con 0x7f3230106610 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.322+0000 7f322e59c640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f321803f840 0x7f3218041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.322+0000 7f322e59c640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f321803f840 0x7f3218041d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32240098c0 tx=0x7f3224006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:28.080 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.326+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3220049020 con 0x7f3230106610 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.510+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f323007b6a0 con 0x7f321803f840 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.510+0000 7f3217fff640 1 -- 172.21.15.107:0/1242766059 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f323007b6a0 con 0x7f321803f840 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f321803f840 msgr2=0x7f3218041d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f321803f840 0x7f3218041d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32240098c0 tx=0x7f3224006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 msgr2=0x7f323007c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f323007c180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32200090b0 tx=0x7f32200307c0 comp rx=0 tx=0).stop 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 shutdown_connections 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f321803f840 0x7f3218041d00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 --2- 172.21.15.107:0/1242766059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3230106610 0x7f323007c180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 >> 172.21.15.107:0/1242766059 conn(0x7f3230101f00 msgr2=0x7f3230102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 shutdown_connections 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:27.513+0000 7f32351d3640 1 -- 172.21.15.107:0/1242766059 wait complete. 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T11:48:28.081 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T11:48:28.082 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-09-21T11:48:28.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:28 smithi107 ceph-mon[30293]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:29.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:29 smithi107 ceph-mon[30293]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:30 smithi107 ceph-mon[30293]: Deploying cephadm binary to smithi107 2024-09-21T11:48:32.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-09-21T11:48:32.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e405d640 1 Processor -- start 2024-09-21T11:48:32.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e405d640 1 -- start start 2024-09-21T11:48:32.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e405d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:32.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e405d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc07b0b0 con 0x7f29dc07c510 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e1df9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e1df9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc07ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60372/0 (socket says 172.21.15.107:60372) 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e1df9640 1 -- 172.21.15.107:0/3431436408 learned_addr learned my addr 172.21.15.107:0/3431436408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.487+0000 7f29e1df9640 1 -- 172.21.15.107:0/3431436408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc07b230 con 0x7f29dc07c510 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.488+0000 7f29e1df9640 1 --2- 172.21.15.107:0/3431436408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc07ab70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009820 tx=0x7f29d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a001ec913cc2a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.488+0000 7f29e0df7640 1 -- 172.21.15.107:0/3431436408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8037070 con 0x7f29dc07c510 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.488+0000 7f29e0df7640 1 -- 172.21.15.107:0/3431436408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f29d8032870 con 0x7f29dc07c510 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.488+0000 7f29e0df7640 1 -- 172.21.15.107:0/3431436408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d80305d0 con 0x7f29dc07c510 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 -- 172.21.15.107:0/3431436408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 msgr2=0x7f29dc07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 --2- 172.21.15.107:0/3431436408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc07ab70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009820 tx=0x7f29d802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 -- 172.21.15.107:0/3431436408 shutdown_connections 2024-09-21T11:48:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 --2- 172.21.15.107:0/3431436408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc07ab70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 -- 172.21.15.107:0/3431436408 >> 172.21.15.107:0/3431436408 conn(0x7f29dc101f00 msgr2=0x7f29dc104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 -- 172.21.15.107:0/3431436408 shutdown_connections 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.489+0000 7f29e405d640 1 -- 172.21.15.107:0/3431436408 wait complete. 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e405d640 1 Processor -- start 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e405d640 1 -- start start 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e405d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e405d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc108760 con 0x7f29dc07c510 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e1df9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e1df9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60380/0 (socket says 172.21.15.107:60380) 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e1df9640 1 -- 172.21.15.107:0/2988849554 learned_addr learned my addr 172.21.15.107:0/2988849554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e1df9640 1 -- 172.21.15.107:0/2988849554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc1a1200 con 0x7f29dc07c510 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e1df9640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc1a0ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29d80090b0 tx=0x7f29d80307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d803f020 con 0x7f29dc07c510 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f29d8030e20 con 0x7f29dc07c510 2024-09-21T11:48:32.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29d8037040 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc1a1490 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.490+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc1a1e50 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.491+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f29d8042020 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.491+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc07abf0 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.491+0000 7f29d2ffd640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f29c803f7f0 0x7f29c8041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.492+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f29d8074790 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.492+0000 7f29e15f8640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f29c803f7f0 0x7f29c8041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.492+0000 7f29e15f8640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f29c803f7f0 0x7f29c8041cb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0098c0 tx=0x7f29cc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.496+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29d8049020 con 0x7f29dc07c510 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:28.676+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) -- 0x7f29dc1a2170 con 0x7f29c803f7f0 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.741+0000 7f29d2ffd640 1 -- 172.21.15.107:0/2988849554 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f29dc1a2170 con 0x7f29c803f7f0 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f29c803f7f0 msgr2=0x7f29c8041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:32.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f29c803f7f0 0x7f29c8041cb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0098c0 tx=0x7f29cc006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 msgr2=0x7f29dc1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc1a0ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29d80090b0 tx=0x7f29d80307c0 comp rx=0 tx=0).stop 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 shutdown_connections 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f29c803f7f0 0x7f29c8041cb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.743+0000 7f29e405d640 1 --2- 172.21.15.107:0/2988849554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29dc07c510 0x7f29dc1a0ca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.744+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 >> 172.21.15.107:0/2988849554 conn(0x7f29dc101f00 msgr2=0x7f29dc1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.744+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 shutdown_connections 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:31.744+0000 7f29e405d640 1 -- 172.21.15.107:0/2988849554 wait complete. 2024-09-21T11:48:32.297 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-09-21T11:48:33.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:32 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:32 smithi107 ceph-mon[30293]: Added host smithi107 2024-09-21T11:48:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:32 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.728+0000 7fdf3c0f3640 1 Processor -- start 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.728+0000 7fdf3c0f3640 1 -- start start 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.729+0000 7fdf3c0f3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf341091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.729+0000 7fdf3c0f3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34109720 con 0x7fdf34079b60 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.729+0000 7fdf39e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf341091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.729+0000 7fdf39e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf341091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60382/0 (socket says 172.21.15.107:60382) 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.729+0000 7fdf39e8f640 1 -- 172.21.15.107:0/583521325 learned_addr learned my addr 172.21.15.107:0/583521325 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.729+0000 7fdf39e8f640 1 -- 172.21.15.107:0/583521325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf341098a0 con 0x7fdf34079b60 2024-09-21T11:48:33.819 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.730+0000 7fdf39e8f640 1 --2- 172.21.15.107:0/583521325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf341091e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009820 tx=0x7fdf3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd1d93c16d5e1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.730+0000 7fdf38e8d640 1 -- 172.21.15.107:0/583521325 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf30037070 con 0x7fdf34079b60 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.730+0000 7fdf38e8d640 1 -- 172.21.15.107:0/583521325 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fdf30032870 con 0x7fdf34079b60 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.730+0000 7fdf38e8d640 1 -- 172.21.15.107:0/583521325 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf300305d0 con 0x7fdf34079b60 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/583521325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 msgr2=0x7fdf341091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 --2- 172.21.15.107:0/583521325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf341091e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009820 tx=0x7fdf3002a270 comp rx=0 tx=0).stop 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/583521325 shutdown_connections 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 --2- 172.21.15.107:0/583521325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf341091e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/583521325 >> 172.21.15.107:0/583521325 conn(0x7fdf34100df0 msgr2=0x7fdf34103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/583521325 shutdown_connections 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/583521325 wait complete. 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 Processor -- start 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.731+0000 7fdf3c0f3640 1 -- start start 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf3c0f3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf3419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:33.820 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf3c0f3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410a840 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf39e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf3419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf39e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf3419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60396/0 (socket says 172.21.15.107:60396) 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf39e8f640 1 -- 172.21.15.107:0/2209100068 learned_addr learned my addr 172.21.15.107:0/2209100068 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf39e8f640 1 -- 172.21.15.107:0/2209100068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419ab80 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf39e8f640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf3419a620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf300090b0 tx=0x7fdf30030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3003f020 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7fdf30030e10 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf30037040 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419ae10 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.732+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3419b850 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.733+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7fdf30042020 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.733+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf34107250 con 0x7fdf34079b60 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.734+0000 7fdf2affd640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fdf2003f840 0x7fdf20041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:33.821 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.734+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf300746e0 con 0x7fdf34079b60 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.734+0000 7fdf3968e640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fdf2003f840 0x7fdf20041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.734+0000 7fdf3968e640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fdf2003f840 0x7fdf20041d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf240098c0 tx=0x7fdf24006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.738+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf30049020 con 0x7fdf34079b60 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.940+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdf3419c030 con 0x7fdf2003f840 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.945+0000 7fdf2affd640 1 -- 172.21.15.107:0/2209100068 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdf3419c030 con 0x7fdf2003f840 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fdf2003f840 msgr2=0x7fdf20041d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fdf2003f840 0x7fdf20041d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf240098c0 tx=0x7fdf24006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 msgr2=0x7fdf3419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf3419a620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf300090b0 tx=0x7fdf30030b50 comp rx=0 tx=0).stop 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 shutdown_connections 2024-09-21T11:48:33.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fdf2003f840 0x7fdf20041d00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:33.823 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 --2- 172.21.15.107:0/2209100068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf34079b60 0x7fdf3419a620 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:33.823 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 >> 172.21.15.107:0/2209100068 conn(0x7fdf34100df0 msgr2=0x7fdf34101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:33.823 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 shutdown_connections 2024-09-21T11:48:33.823 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:32.947+0000 7fdf3c0f3640 1 -- 172.21.15.107:0/2209100068 wait complete. 2024-09-21T11:48:33.823 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-09-21T11:48:34.086 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:33 smithi107 ceph-mon[30293]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:34.086 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:33 smithi107 ceph-mon[30293]: Saving service mon spec with placement count:5 2024-09-21T11:48:34.086 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:33 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:35 smithi107 ceph-mon[30293]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:35 smithi107 ceph-mon[30293]: Saving service mgr spec with placement count:2 2024-09-21T11:48:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:35 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:35 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:35.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T11:48:35.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.303+0000 7f5afda12640 1 Processor -- start 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.303+0000 7f5afda12640 1 -- start start 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.303+0000 7f5afda12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8109240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.303+0000 7f5afda12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8109780 con 0x7f5af8079b00 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.304+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.304+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39392/0 (socket says 172.21.15.107:39392) 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.304+0000 7f5af6ffd640 1 -- 172.21.15.107:0/2985935934 learned_addr learned my addr 172.21.15.107:0/2985935934 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.304+0000 7f5af6ffd640 1 -- 172.21.15.107:0/2985935934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af8109900 con 0x7f5af8079b00 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5af6ffd640 1 --2- 172.21.15.107:0/2985935934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8109240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009820 tx=0x7f5ae002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90dc4b809a7d032 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5af5ffb640 1 -- 172.21.15.107:0/2985935934 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae0037070 con 0x7f5af8079b00 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5af5ffb640 1 -- 172.21.15.107:0/2985935934 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f5ae0032870 con 0x7f5af8079b00 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5af5ffb640 1 -- 172.21.15.107:0/2985935934 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae00305d0 con 0x7f5af8079b00 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 -- 172.21.15.107:0/2985935934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 msgr2=0x7f5af8109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:35.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 --2- 172.21.15.107:0/2985935934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8109240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009820 tx=0x7f5ae002a270 comp rx=0 tx=0).stop 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 -- 172.21.15.107:0/2985935934 shutdown_connections 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 --2- 172.21.15.107:0/2985935934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8109240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 -- 172.21.15.107:0/2985935934 >> 172.21.15.107:0/2985935934 conn(0x7f5af8100df0 msgr2=0x7f5af8103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 -- 172.21.15.107:0/2985935934 shutdown_connections 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.305+0000 7f5afda12640 1 -- 172.21.15.107:0/2985935934 wait complete. 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5afda12640 1 Processor -- start 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5afda12640 1 -- start start 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5afda12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8106890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5afda12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5af810a840 con 0x7f5af8079b00 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5af6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39396/0 (socket says 172.21.15.107:39396) 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.306+0000 7f5af6ffd640 1 -- 172.21.15.107:0/2426050153 learned_addr learned my addr 172.21.15.107:0/2426050153 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5af6ffd640 1 -- 172.21.15.107:0/2426050153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af8106df0 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5af6ffd640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8106890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00090b0 tx=0x7f5ae0030b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae003f020 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 972+0+0 (secure 0 0 0) 0x7f5ae0030e10 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af81051e0 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae0037040 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.307+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af81057c0 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.308+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af81072b0 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.308+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f5ae0042020 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.308+0000 7f5afca10640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5ae403f7f0 0x7f5ae4041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.309+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5ae0074820 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.309+0000 7f5af67fc640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5ae403f7f0 0x7f5ae4041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.309+0000 7f5af67fc640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5ae403f7f0 0x7f5ae4041cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80098c0 tx=0x7f5ae8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.313+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ae0049020 con 0x7f5af8079b00 2024-09-21T11:48:35.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.513+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5af8105e70 con 0x7f5ae403f7f0 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.518+0000 7f5afca10640 1 -- 172.21.15.107:0/2426050153 <== mgr.14118 v2:172.21.15.107:6800/1807990149 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5af8105e70 con 0x7f5ae403f7f0 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5ae403f7f0 msgr2=0x7f5ae4041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5ae403f7f0 0x7f5ae4041cb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80098c0 tx=0x7f5ae8006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 msgr2=0x7f5af8106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8106890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00090b0 tx=0x7f5ae0030b50 comp rx=0 tx=0).stop 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 shutdown_connections 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f5ae403f7f0 0x7f5ae4041cb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 --2- 172.21.15.107:0/2426050153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5af8079b00 0x7f5af8106890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 >> 172.21.15.107:0/2426050153 conn(0x7f5af8100df0 msgr2=0x7f5af8101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 shutdown_connections 2024-09-21T11:48:35.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:34.520+0000 7f5afda12640 1 -- 172.21.15.107:0/2426050153 wait complete. 2024-09-21T11:48:37.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:37 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:37.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/861583833' entity='client.admin' 2024-09-21T11:48:37.356 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.327+0000 7f0d3f6dc640 1 Processor -- start 2024-09-21T11:48:37.356 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.327+0000 7f0d3f6dc640 1 -- start start 2024-09-21T11:48:37.356 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.327+0000 7f0d3f6dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:37.356 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.327+0000 7f0d3f6dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38105ed0 con 0x7f0d381054e0 2024-09-21T11:48:37.356 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.327+0000 7f0d3d478640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.328+0000 7f0d3d478640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39400/0 (socket says 172.21.15.107:39400) 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.328+0000 7f0d3d478640 1 -- 172.21.15.107:0/3916053077 learned_addr learned my addr 172.21.15.107:0/3916053077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.328+0000 7f0d3d478640 1 -- 172.21.15.107:0/3916053077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d38106050 con 0x7f0d381054e0 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.328+0000 7f0d3d478640 1 --2- 172.21.15.107:0/3916053077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381058e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009820 tx=0x7f0d2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c3a630209aea86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d2bfff640 1 -- 172.21.15.107:0/3916053077 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d2c037070 con 0x7f0d381054e0 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d2bfff640 1 -- 172.21.15.107:0/3916053077 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0d2c032890 con 0x7f0d381054e0 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d2bfff640 1 -- 172.21.15.107:0/3916053077 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d2c030550 con 0x7f0d381054e0 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/3916053077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 msgr2=0x7f0d381058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 --2- 172.21.15.107:0/3916053077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381058e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009820 tx=0x7f0d2c02a270 comp rx=0 tx=0).stop 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/3916053077 shutdown_connections 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 --2- 172.21.15.107:0/3916053077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381058e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/3916053077 >> 172.21.15.107:0/3916053077 conn(0x7f0d38100db0 msgr2=0x7f0d381031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/3916053077 shutdown_connections 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.329+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/3916053077 wait complete. 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.330+0000 7f0d3f6dc640 1 Processor -- start 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.330+0000 7f0d3f6dc640 1 -- start start 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.330+0000 7f0d3f6dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.330+0000 7f0d3f6dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d38109960 con 0x7f0d381054e0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3d478640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3d478640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39412/0 (socket says 172.21.15.107:39412) 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3d478640 1 -- 172.21.15.107:0/861583833 learned_addr learned my addr 172.21.15.107:0/861583833 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3d478640 1 -- 172.21.15.107:0/861583833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d38199a30 con 0x7f0d381054e0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3d478640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381994d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c032860 tx=0x7f0d2c030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d2c037040 con 0x7f0d381054e0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0d2c040070 con 0x7f0d381054e0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d38199cc0 con 0x7f0d381054e0 2024-09-21T11:48:37.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d2c03bcc0 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.331+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3819a680 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.332+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f0d2c045460 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.332+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d38109ae0 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.332+0000 7f0d2a7fc640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f0d2403f840 0x7f0d24041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.332+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0d2c0743e0 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.333+0000 7f0d3cc77640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f0d2403f840 0x7f0d24041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.333+0000 7f0d3cc77640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f0d2403f840 0x7f0d24041d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0d340098c0 tx=0x7f0d34006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.337+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d2c030d50 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.547+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0d381058e0 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.549+0000 7f0d2a7fc640 1 -- 172.21.15.107:0/861583833 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0d2c0443c0 con 0x7f0d381054e0 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f0d2403f840 msgr2=0x7f0d24041d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f0d2403f840 0x7f0d24041d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0d340098c0 tx=0x7f0d34006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 msgr2=0x7f0d381994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:37.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381994d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c032860 tx=0x7f0d2c030a80 comp rx=0 tx=0).stop 2024-09-21T11:48:37.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 shutdown_connections 2024-09-21T11:48:37.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f0d2403f840 0x7f0d24041d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:37.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 --2- 172.21.15.107:0/861583833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d381054e0 0x7f0d381994d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:37.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 >> 172.21.15.107:0/861583833 conn(0x7f0d38100db0 msgr2=0x7f0d38101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:37.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 shutdown_connections 2024-09-21T11:48:37.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:36.553+0000 7f0d3f6dc640 1 -- 172.21.15.107:0/861583833 wait complete. 2024-09-21T11:48:38.941 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.755+0000 7f2368273640 1 Processor -- start 2024-09-21T11:48:38.941 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.755+0000 7f2368273640 1 -- start start 2024-09-21T11:48:38.941 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.756+0000 7f2368273640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f2360109260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:38.941 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.756+0000 7f2368273640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23601097a0 con 0x7f2360079ae0 2024-09-21T11:48:38.941 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.756+0000 7f236600f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f2360109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:38.941 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.756+0000 7f236600f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f2360109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39414/0 (socket says 172.21.15.107:39414) 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.756+0000 7f236600f640 1 -- 172.21.15.107:0/1673247897 learned_addr learned my addr 172.21.15.107:0/1673247897 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.756+0000 7f236600f640 1 -- 172.21.15.107:0/1673247897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2360109920 con 0x7f2360079ae0 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.757+0000 7f236600f640 1 --2- 172.21.15.107:0/1673247897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f2360109260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f235c009820 tx=0x7f235c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248751d44dfdcc0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.757+0000 7f236500d640 1 -- 172.21.15.107:0/1673247897 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c037070 con 0x7f2360079ae0 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.757+0000 7f236500d640 1 -- 172.21.15.107:0/1673247897 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f235c032890 con 0x7f2360079ae0 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.757+0000 7f236500d640 1 -- 172.21.15.107:0/1673247897 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c030550 con 0x7f2360079ae0 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.757+0000 7f2368273640 1 -- 172.21.15.107:0/1673247897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 msgr2=0x7f2360109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.757+0000 7f2368273640 1 --2- 172.21.15.107:0/1673247897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f2360109260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f235c009820 tx=0x7f235c02a270 comp rx=0 tx=0).stop 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 -- 172.21.15.107:0/1673247897 shutdown_connections 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 --2- 172.21.15.107:0/1673247897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f2360109260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 -- 172.21.15.107:0/1673247897 >> 172.21.15.107:0/1673247897 conn(0x7f2360100df0 msgr2=0x7f2360103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 -- 172.21.15.107:0/1673247897 shutdown_connections 2024-09-21T11:48:38.942 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 -- 172.21.15.107:0/1673247897 wait complete. 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 Processor -- start 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 -- start start 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f23601068b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.758+0000 7f2368273640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f236010a910 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f236600f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f23601068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f236600f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f23601068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39426/0 (socket says 172.21.15.107:39426) 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f236600f640 1 -- 172.21.15.107:0/2272103862 learned_addr learned my addr 172.21.15.107:0/2272103862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f236600f640 1 -- 172.21.15.107:0/2272103862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2360106e10 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f236600f640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f23601068b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f235c0097f0 tx=0x7f235c030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c037040 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f235c040070 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2360105200 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f235c03bd00 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.759+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2360105780 con 0x7f2360079ae0 2024-09-21T11:48:38.943 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.760+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f235c045460 con 0x7f2360079ae0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.760+0000 7f23577fe640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f234c03f7f0 0x7f234c041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.760+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2360107040 con 0x7f2360079ae0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.761+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f235c0744a0 con 0x7f2360079ae0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.761+0000 7f236580e640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f234c03f7f0 0x7f234c041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.761+0000 7f236580e640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f234c03f7f0 0x7f234c041cb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23500098c0 tx=0x7f2350006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.765+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f235c030c60 con 0x7f2360079ae0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.943+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f2360107260 con 0x7f2360079ae0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.946+0000 7f23577fe640 1 -- 172.21.15.107:0/2272103862 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f235c073df0 con 0x7f2360079ae0 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f234c03f7f0 msgr2=0x7f234c041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f234c03f7f0 0x7f234c041cb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23500098c0 tx=0x7f2350006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 msgr2=0x7f23601068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f23601068b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f235c0097f0 tx=0x7f235c030ad0 comp rx=0 tx=0).stop 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 shutdown_connections 2024-09-21T11:48:38.944 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f234c03f7f0 0x7f234c041cb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:38.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 --2- 172.21.15.107:0/2272103862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2360079ae0 0x7f23601068b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:38.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 >> 172.21.15.107:0/2272103862 conn(0x7f2360100df0 msgr2=0x7f2360101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:38.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 shutdown_connections 2024-09-21T11:48:38.945 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:37.948+0000 7f2368273640 1 -- 172.21.15.107:0/2272103862 wait complete. 2024-09-21T11:48:38.945 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-09-21T11:48:39.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2272103862' entity='client.admin' 2024-09-21T11:48:39.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:38 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:40.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:39 smithi107 ceph-mon[30293]: from='mgr.14118 172.21.15.107:0/2556961702' entity='mgr.y' 2024-09-21T11:48:40.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:39 smithi107 ceph-mon[30293]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:48:40.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1398936797' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T11:48:40.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:40 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ignoring --setuser ceph since I am not root 2024-09-21T11:48:40.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:40 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ignoring --setgroup ceph since I am not root 2024-09-21T11:48:41.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef3443640 1 Processor -- start 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef3443640 1 -- start start 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef3443640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef3443640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec107000 con 0x7f2eec106610 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef11df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef11df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39442/0 (socket says 172.21.15.107:39442) 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef11df640 1 -- 172.21.15.107:0/2813825310 learned_addr learned my addr 172.21.15.107:0/2813825310 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.367+0000 7f2ef11df640 1 -- 172.21.15.107:0/2813825310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec107180 con 0x7f2eec106610 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.368+0000 7f2ef11df640 1 --2- 172.21.15.107:0/2813825310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec106a10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009820 tx=0x7f2ee802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8ffb019b1467bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.368+0000 7f2ee3fff640 1 -- 172.21.15.107:0/2813825310 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee8037070 con 0x7f2eec106610 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.368+0000 7f2ee3fff640 1 -- 172.21.15.107:0/2813825310 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2ee8032890 con 0x7f2eec106610 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.368+0000 7f2ee3fff640 1 -- 172.21.15.107:0/2813825310 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee8030550 con 0x7f2eec106610 2024-09-21T11:48:41.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 -- 172.21.15.107:0/2813825310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 msgr2=0x7f2eec106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 --2- 172.21.15.107:0/2813825310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec106a10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009820 tx=0x7f2ee802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 -- 172.21.15.107:0/2813825310 shutdown_connections 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 --2- 172.21.15.107:0/2813825310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec106a10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 -- 172.21.15.107:0/2813825310 >> 172.21.15.107:0/2813825310 conn(0x7f2eec101f00 msgr2=0x7f2eec104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 -- 172.21.15.107:0/2813825310 shutdown_connections 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.369+0000 7f2ef3443640 1 -- 172.21.15.107:0/2813825310 wait complete. 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef3443640 1 Processor -- start 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef3443640 1 -- start start 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef3443640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef3443640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10aa90 con 0x7f2eec106610 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef11df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef11df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39452/0 (socket says 172.21.15.107:39452) 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef11df640 1 -- 172.21.15.107:0/1398936797 learned_addr learned my addr 172.21.15.107:0/1398936797 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:41.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef11df640 1 -- 172.21.15.107:0/1398936797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec19ab30 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef11df640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec19a5d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80097f0 tx=0x7f2ee8030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee8037040 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2ee8040070 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.370+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec19adc0 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.371+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee803bd40 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.371+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec19b780 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.372+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 51842+0+0 (secure 0 0 0) 0x7f2ee803b430 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.372+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec10ac10 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.372+0000 7f2ee27fc640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f2ed803f840 0x7f2ed8041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.372+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ee8074360 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.372+0000 7f2ef09de640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f2ed803f840 0x7f2ed8041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.372+0000 7f2ef09de640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f2ed803f840 0x7f2ed8041d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098c0 tx=0x7f2edc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.377+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ee80313a0 con 0x7f2eec106610 2024-09-21T11:48:41.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:39.615+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f2eec106a10 con 0x7f2eec106610 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.260+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v10) ==== 88+0+0 (secure 0 0 0) 0x7f2ee803ae00 con 0x7f2eec106610 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.261+0000 7f2ee27fc640 1 -- 172.21.15.107:0/1398936797 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 10) ==== 51855+0+0 (secure 0 0 0) 0x7f2ee8045980 con 0x7f2eec106610 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.262+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f2ed803f840 msgr2=0x7f2ed8041d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.262+0000 7f2ef3443640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f2ed803f840 0x7f2ed8041d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098c0 tx=0x7f2edc006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 msgr2=0x7f2eec19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec19a5d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80097f0 tx=0x7f2ee8030ad0 comp rx=0 tx=0).stop 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 shutdown_connections 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7f2ed803f840 0x7f2ed8041d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 --2- 172.21.15.107:0/1398936797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eec106610 0x7f2eec19a5d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 >> 172.21.15.107:0/1398936797 conn(0x7f2eec101f00 msgr2=0x7f2eec102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 shutdown_connections 2024-09-21T11:48:41.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:40.263+0000 7f2ef3443640 1 -- 172.21.15.107:0/1398936797 wait complete. 2024-09-21T11:48:41.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1398936797' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T11:48:41.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:41 smithi107 ceph-mon[30293]: mgrmap e10: y(active, since 21s) 2024-09-21T11:48:42.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1919542167' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.643+0000 7fcd614f2640 1 Processor -- start 2024-09-21T11:48:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.643+0000 7fcd614f2640 1 -- start start 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.643+0000 7fcd614f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c1058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.643+0000 7fcd614f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c105ee0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.644+0000 7fcd5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c1058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.644+0000 7fcd5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c1058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39466/0 (socket says 172.21.15.107:39466) 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.644+0000 7fcd5affd640 1 -- 172.21.15.107:0/954462784 learned_addr learned my addr 172.21.15.107:0/954462784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.644+0000 7fcd5affd640 1 -- 172.21.15.107:0/954462784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c106060 con 0x7fcd5c1054f0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd5affd640 1 --2- 172.21.15.107:0/954462784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c1058f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009820 tx=0x7fcd4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8565b7ed1c87c5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd59ffb640 1 -- 172.21.15.107:0/954462784 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd4c037070 con 0x7fcd5c1054f0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd59ffb640 1 -- 172.21.15.107:0/954462784 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fcd4c032890 con 0x7fcd5c1054f0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd59ffb640 1 -- 172.21.15.107:0/954462784 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd4c030550 con 0x7fcd5c1054f0 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 -- 172.21.15.107:0/954462784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 msgr2=0x7fcd5c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 --2- 172.21.15.107:0/954462784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c1058f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009820 tx=0x7fcd4c02a270 comp rx=0 tx=0).stop 2024-09-21T11:48:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 -- 172.21.15.107:0/954462784 shutdown_connections 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 --2- 172.21.15.107:0/954462784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c1058f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 -- 172.21.15.107:0/954462784 >> 172.21.15.107:0/954462784 conn(0x7fcd5c100e00 msgr2=0x7fcd5c103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 -- 172.21.15.107:0/954462784 shutdown_connections 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.645+0000 7fcd614f2640 1 -- 172.21.15.107:0/954462784 wait complete. 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd614f2640 1 Processor -- start 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd614f2640 1 -- start start 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd614f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c07a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd614f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c109970 con 0x7fcd5c1054f0 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c07a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c07a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39474/0 (socket says 172.21.15.107:39474) 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.646+0000 7fcd5affd640 1 -- 172.21.15.107:0/1919542167 learned_addr learned my addr 172.21.15.107:0/1919542167 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd5affd640 1 -- 172.21.15.107:0/1919542167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c07a5e0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd5affd640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c07a080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c032860 tx=0x7fcd4c030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd4c037040 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fcd4c040070 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd4c03bd60 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c19d2a0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.647+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c19d780 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.648+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 10) ==== 51855+0+0 (secure 0 0 0) 0x7fcd4c045460 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.648+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c109af0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.649+0000 7fcd43fff640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 0x7fcd44041d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.649+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd4c0744c0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.649+0000 7fcd5a7fc640 1 -- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 msgr2=0x7fcd44041d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1807990149 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.649+0000 7fcd5a7fc640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 0x7fcd44041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.653+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd4c030dc0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.850+0000 7fcd5a7fc640 1 -- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 msgr2=0x7fcd44041d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1807990149 2024-09-21T11:48:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.850+0000 7fcd5a7fc640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 0x7fcd44041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.885+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcd5c19da30 con 0x7fcd5c1054f0 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.885+0000 7fcd43fff640 1 -- 172.21.15.107:0/1919542167 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v10) ==== 56+0+89 (secure 0 0 0) 0x7fcd4c0443c0 con 0x7fcd5c1054f0 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 msgr2=0x7fcd44041d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 0x7fcd44041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 msgr2=0x7fcd5c07a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c07a080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c032860 tx=0x7fcd4c030a80 comp rx=0 tx=0).stop 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 shutdown_connections 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fcd4403f8a0 0x7fcd44041d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 --2- 172.21.15.107:0/1919542167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd5c1054f0 0x7fcd5c07a080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 >> 172.21.15.107:0/1919542167 conn(0x7fcd5c100e00 msgr2=0x7fcd5c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 shutdown_connections 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:41.888+0000 7fcd614f2640 1 -- 172.21.15.107:0/1919542167 wait complete. 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-09-21T11:48:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 10... 2024-09-21T11:48:45.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: Active manager daemon y restarted 2024-09-21T11:48:45.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: Activating manager daemon y 2024-09-21T11:48:45.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T11:48:45.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: mgrmap e11: y(active, starting, since 0.00362853s) 2024-09-21T11:48:45.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:48:45.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:48:45.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:48:45.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:48:45.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:48:45.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:45 smithi107 ceph-mon[30293]: Manager daemon y is now available 2024-09-21T11:48:46.642 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:46.642 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:48:46.642 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:46] ENGINE Bus STARTING 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-mon[30293]: mgrmap e12: y(active, since 1.00583s) 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:48:46.535+0000 7fb5bc0f6640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:11:48:46] ENGINE Error in HTTPServer.serve 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: Traceback (most recent call last): 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._connections.run(self.expiration_interval) 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._run(expiration_interval) 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:48:46.643 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: s = self.context.wrap_socket( 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: return self.sslsocket_class._create( 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self.do_handshake() 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._sslobj.do_handshake() 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:48:46.644 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:48:46 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd94a44e640 1 Processor -- start 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd94a44e640 1 -- start start 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd94a44e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd9441091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd94a44e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd944109720 con 0x7fd944079b60 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd943fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd9441091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd943fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd9441091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39484/0 (socket says 172.21.15.107:39484) 2024-09-21T11:48:47.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.909+0000 7fd943fff640 1 -- 172.21.15.107:0/375281741 learned_addr learned my addr 172.21.15.107:0/375281741 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.910+0000 7fd943fff640 1 -- 172.21.15.107:0/375281741 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9441098a0 con 0x7fd944079b60 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.911+0000 7fd943fff640 1 --2- 172.21.15.107:0/375281741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd9441091e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd934009820 tx=0x7fd93402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4d29a614a1c188 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.911+0000 7fd942ffd640 1 -- 172.21.15.107:0/375281741 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934037070 con 0x7fd944079b60 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.911+0000 7fd942ffd640 1 -- 172.21.15.107:0/375281741 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd934032890 con 0x7fd944079b60 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.911+0000 7fd942ffd640 1 -- 172.21.15.107:0/375281741 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934030550 con 0x7fd944079b60 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 -- 172.21.15.107:0/375281741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 msgr2=0x7fd9441091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 --2- 172.21.15.107:0/375281741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd9441091e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd934009820 tx=0x7fd93402a270 comp rx=0 tx=0).stop 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 -- 172.21.15.107:0/375281741 shutdown_connections 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 --2- 172.21.15.107:0/375281741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd9441091e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 -- 172.21.15.107:0/375281741 >> 172.21.15.107:0/375281741 conn(0x7fd944100df0 msgr2=0x7fd944103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 -- 172.21.15.107:0/375281741 shutdown_connections 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 -- 172.21.15.107:0/375281741 wait complete. 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 Processor -- start 2024-09-21T11:48:47.364 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.912+0000 7fd94a44e640 1 -- start start 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd94a44e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd94419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd94a44e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410a890 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd943fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd94419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd943fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd94419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39500/0 (socket says 172.21.15.107:39500) 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd943fff640 1 -- 172.21.15.107:0/571584542 learned_addr learned my addr 172.21.15.107:0/571584542 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd943fff640 1 -- 172.21.15.107:0/571584542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94419ab00 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd943fff640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd94419a5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9340097f0 tx=0x7fd934030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934037040 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.913+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd934040070 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.914+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94419ad90 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.914+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd93403bc90 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.914+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94419b7d0 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.915+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 10) ==== 51855+0+0 (secure 0 0 0) 0x7fd934045460 con 0x7fd944079b60 2024-09-21T11:48:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.915+0000 7fd9417fa640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.915+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 --> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd92c042470 con 0x7fd92c03f8a0 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.915+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd934074470 con 0x7fd944079b60 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.915+0000 7fd9437fe640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 msgr2=0x7fd92c041d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1807990149 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:42.915+0000 7fd9437fe640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:43.116+0000 7fd9437fe640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 msgr2=0x7fd92c041d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1807990149 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:43.116+0000 7fd9437fe640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:43.517+0000 7fd9437fe640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 msgr2=0x7fd92c041d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1807990149 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:43.517+0000 7fd9437fe640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:44.317+0000 7fd9437fe640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 msgr2=0x7fd92c041d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1807990149 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:44.317+0000 7fd9437fe640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:45.320+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 11) ==== 51622+0+0 (secure 0 0 0) 0x7fd934072f20 con 0x7fd944079b60 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:45.320+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 msgr2=0x7fd92c041d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:48:47.366 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:45.320+0000 7fd9417fa640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.321+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 12) ==== 51749+0+0 (secure 0 0 0) 0x7fd9340066a0 con 0x7fd944079b60 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.321+0000 7fd9417fa640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd92c043340 0x7fd92c045730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.321+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd92c042470 con 0x7fd92c043340 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.322+0000 7fd9437fe640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd92c043340 0x7fd92c045730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.323+0000 7fd9437fe640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd92c043340 0x7fd92c045730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd938003980 tx=0x7fd9380092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.323+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd92c042470 con 0x7fd92c043340 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.330+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd9441071d0 con 0x7fd92c043340 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd9417fa640 1 -- 172.21.15.107:0/571584542 <== mgr.14152 v2:172.21.15.107:6800/2130999972 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd9441071d0 con 0x7fd92c043340 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd92c043340 msgr2=0x7fd92c045730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd92c043340 0x7fd92c045730 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd938003980 tx=0x7fd9380092b0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 msgr2=0x7fd94419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd94419a5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9340097f0 tx=0x7fd934030ad0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 shutdown_connections 2024-09-21T11:48:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd92c043340 0x7fd92c045730 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:6800/1807990149,v1:172.21.15.107:6801/1807990149] conn(0x7fd92c03f8a0 0x7fd92c041d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 --2- 172.21.15.107:0/571584542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd944079b60 0x7fd94419a5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 >> 172.21.15.107:0/571584542 conn(0x7fd944100df0 msgr2=0x7fd944101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 shutdown_connections 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:46.331+0000 7fd94a44e640 1 -- 172.21.15.107:0/571584542 wait complete. 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 10 is available 2024-09-21T11:48:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:46] ENGINE Serving on http://172.21.15.107:8765 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:46] ENGINE Serving on https://172.21.15.107:7150 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:46] ENGINE Bus STARTED 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: [21/Sep/2024:11:48:46] ENGINE Error in HTTPServer.serve 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-21T11:48:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:48:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:47 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:48:49.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:48 smithi107 ceph-mon[30293]: mgrmap e13: y(active, since 2s) 2024-09-21T11:48:49.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:48 smithi107 ceph-mon[30293]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:49.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFETCCAvmgAwIBAgIURre9Ti6RrhsKGSVrw0FZ5OGybl4wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTExNDg0OFoXDTM0\nMDkyMjExNDg0OFowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjEwNzCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAKNsJ3OZ2iNyQDsKvCnj53ogiAACArG7AGF6\n7kixPf628LumAMU+mKxMIFrZDXp6Tptc/GK3bm/+CEhh925PBuSGwRMxW/GH1zpL\ng35+qGaRVCPm0Ti3bmUL7FZ8TYBGnaN6LJbJPjbsqIZBOJKQh/flyjxGGUrt3F3F\nTH97qGuZRqIMEkGNxi63M0zTMVYCIZJQam2E7mxHcwBLS78xcz3pySIAwvqsFeTn\n81IzDY3BLPAoZFV/tqo8PJIltAgwz1qZedeJfMc/pi8WjvK0nyE9lPK+Qbhfiatz\nqCNeKsfB4kOu4f6k4lu7ZitjzhXIoDMFENt9CawU5d0GgOc/60HCeFM2XbvhzBP2\nWtC8nJgKcet8uOW+0C0BQj6g/UA5xgx6Ps7L+WbvfkvslHj53R79z2C+SthWuE1Z\nTSEvE8cg8nacmEigJbj1lg6+IrjKo7S3IpPUkVp7ADKRwF3TLnYfY4J4kvMiyvRS\n56xFyC4RGQHuJDV1laZxUvBHxM65UNguWVn+u2yAY0XKKoRgmJBw0nNHBgk9O1Wc\nIi3wV7pRXjEunhoACKKPpM6g6w/jlXloa4ys8cNKp/nd6ojQw9KCdBs7XtqikqlH\nSB5WhAMrK7j9hM9KKafuw9C14XytjFHR7Cmbbs/qUZEe5nLXJDhLISC+skKBVily\nAeqBEjPJAgMBAAGjVDBSMEIGA1UdEQQ7MDmCHnNtaXRoaTEwNy5mcm9udC5zZXBp\nYS5jZXBoLmNvbYIRZGFzaGJvYXJkX3NlcnZlcnOHBKwVD2swDAYDVR0TAQH/BAIw\nADANBgkqhkiG9w0BAQsFAAOCAgEAa6NbVv9IRO0f/1Siq4SQSz41ReMvYz9u3GIp\nN18FpEd1bZY+HXSblnsZBGnDjupvoiZ6LdwJrNFojevdp5L5AbpFaX5jmCiUU6k3\nwEszgoEVCQ9O6pEz6WLYnWLiFNw1fvKtRRVp9BZUsliW/FEs7/hJ47/eTq06tXD2\nZ6+Ml/B9ncAkkF+HvFLjcendb/p23RQPIkrGIoAw1vlZNubWbJSgiPDVwHtJCNkN\nFb+faMs114vjk9TiFezK8HYo3v2fWFqL6l4T8CajOET9HSbW3oHMZnTV53eNJsKm\ngEf8QuupABL1QZR3vIiMOVcEW5FJQcc6B3PTijRPJQCduTfx2A48y3r+JED2EIwo\n6ktLsjumLk3gQVY0z1VsjBOubzO3TQZtncTtW9te7ECBTdgn7ccC6BkfXzHGwf82\n9xhrP46CJD9V/1q+okXrKotheAl772E2CJ2tmWwfuytxs6sBuhpvwqfnLpim8PS7\n1533PcQ6H5GrIx8Ep5fBS5r9XqyRWKGRvoD/d5mPD239UpxicDAElqJaZmwaSICY\nuoKZe74paO4u4O+ULNAQFJMkcwYonf3VUgyx61nTy1ZWHhdmZAHuECYF+o2Ruf1d\n6COhCWNyTkHnT9X5dyo4t7LoIsWwYfcKtM12XMLpum3ZgtR6l3J33TMuEe8ZAk7Y\nvz8NSkQ=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAo2wnc5naI3JAOwq8KePneiCIAAICsbsAYXruSLE9/rbwu6YA\nxT6YrEwgWtkNenpOm1z8Yrdub/4ISGH3bk8G5IbBEzFb8YfXOkuDfn6oZpFUI+bR\nOLduZQvsVnxNgEado3oslsk+NuyohkE4kpCH9+XKPEYZSu3cXcVMf3uoa5lGogwS\nQY3GLrczTNMxVgIhklBqbYTubEdzAEtLvzFzPenJIgDC+qwV5OfzUjMNjcEs8Chk\nVX+2qjw8kiW0CDDPWpl514l8xz+mLxaO8rSfIT2U8r5BuF+Jq3OoI14qx8HiQ67h\n/qTiW7tmK2POFcigMwUQ230JrBTl3QaA5z/rQcJ4UzZdu+HME/Za0LycmApx63y4\n5b7QLQFCPqD9QDnGDHo+zsv5Zu9+S+yUePndHv3PYL5K2Fa4TVlNIS8TxyDydpyY\nSKAluPWWDr4iuMqjtLcik9SRWnsAMpHAXdMudh9jgniS8yLK9FLnrEXILhEZAe4k\nNXWVpnFS8EfEzrlQ2C5ZWf67bIBjRcoqhGCYkHDSc0cGCT07VZwiLfBXulFeMS6e\nGgAIoo+kzqDrD+OVeWhrjKzxw0qn+d3qiNDD0oJ0Gzte2qKSqUdIHlaEAysruP2E\nz0opp+7D0LXhfK2MUdHsKZtuz+pRkR7mctckOEshIL6yQoFWKXIB6oESM8kCAwEA\nAQKCAgAJrY1iDxPQKtNTM8nsJKkoDj5jmrrgXeHSRAqRvz5bw+d6g2STCceWl40I\neEdwMAdQ89cl/PUzOGzy5WtW6uLufGz5dyA56+ii9Qcg4f0U0ukz1ZbhGBrXbNb2\nkbawwK3T+TekB9+xdr8tsTXZlynYNFFvU5rqmS7GVll3CMKS+bEpRlP78hqRGGwX\nMgAfVvG4RZ8h5IAOpUYSF8Nw+4uWuG4+/2LMn82NoG9kWC3mrdCcmdXESvONPhM7\nt/QjEonrFID20tdzAMnVis1VIf11BV25aOtBMRY48zjbPFfF++1LgxMy5n6Zvl24\n9tiE6XxUPSnwB6X/nP62YczXNQ24X0BHEZLOxKDdviyUVHf7i7+tqGde2/H2kpnw\ne68BsgFu6ug090KrciDevq0e7o0velTUNCN9nvoirJ9HAfakcquR1OklcAqWcnHI\nKt45dIEzpXhJ2X+cYWYVFBq4njnWWXkQkAwqOMufOTgJH1sgcP6D0+dpMyZ0PPZm\nZZDvRjTPE/dXgY/yp5zkJ0/rEQPgLpNjG6n8SApKiNOXKYVDDQc2us2kcQts44hY\nTP2dFxvbR3vbjOUp5i8COOMVYaWHs2jPGdMufKFt7JRJlrCnzt7wEBfEtTAdEMpr\nq//KzZOVsQdCc9lrYnEg3W8D7rytA22wxv1Q+gTZSvoQou+mIQKCAQEA42nxOVbe\nshYFWPuFZwfsHX/hLwjd9TopHGhhRbh9RHoxXRIZgYPodoWZdUKTbq0Zmu/jACZT\nnOYvYdYGbYqewtjcEnrDtlpppUebBZisFC5kKaDfN4AUuQgtk+XE1N5NnLIFUb1e\n3wt3i97dG3jQjqsGyDYa2L5kiKWdX5rJCkW3xQ8Jd6ADeHj7mbflyH53Krk3rz7q\ndkLa2ZxxBx3fMBW6P6kZb3hKwcBUXvASmgJX33gEmYNJ+EhxYj/cUYSqw1wSiOIb\n9h7+QxMVQWlYUFt6s7Ngzy1ciVB6NDXHMdcmuJn/2ItBzhdBdFkJFfN5cA8IJO17\n4waecoG2j4pqoQKCAQEAt/cA5X7SvGMlMfnWwS66osyySEbp5RYu3OCWSNM8RlhA\n1dBdK4wyC9HDoy8g42jxW6FQJl0P7+ZGQFrB/ZdHbkRx2gC8z3lv+ngEH2lR/0KB\nzjWwoxKM2p2dPYTNOI8YiTIxBJX+AGV5o2xrcMWpcDHJngtiKtGCgYfkd4gZrUo/\nkU8IVRRJ6xy9dTb9vwfSrt0W8taIpmXgwFqd/jsph4U/FrTfaEd2F06tI/MNZmci\neAxWrgMDc97eDHQ7C/ZWQrvBQOB4/7+orJ1OvOpq549/2+xGKCa3CINxY//OB3Ov\nZCf1RCeaOR4Vrt073NZMKRR5PK8vMVj+9Z26qLMgKQKCAQEAtQX4BvSngIsW5qYQ\notsjkLfiwFYGmNYbM4nDJISe8otyD+wAcaBWw0HFzhmY0ljoMPUTdWMM27A/t1lu\n9HOLvP5y8CLY96cwwWgmdmX5VHc5NAnipYNG+6MO5NPZwrjSfxEWOuLLSjEOkEBv\nMRcSrP5m/6hR5oZn8H4kr52+unaUukAmARtWp+dmIA7K0rYYleGp+c3TRSAipWZq\ncofMybIB9/rbTGgs2d4HSys/I2kKGdCnmKtxjjncR6Ed96XraGc6OcwDBf/V1f6y\nxMbejdjD65e2UDSDnKp7HGRSk9H3+KpdhjoTv8GzULIvEVImMJVGnl6MZ0M1jWDD\nua8tAQKCAQAo4bdNSNXED7RgWskivSO0shCch0XLJxba9v3uySKH72zZFyjdWJZ2\nkfML/JkucE/SsRKCtAOqqF3MR1783/V7x7IVZDyRTsdXvfwMjvct5rW6AaxzUNgE\nbSpYVX7VptcpBgJmpn5/0A0sCBgIQ6TXzfikKptyMIT0DCL1fUVXgTyHKi0L2ypi\nGUzxE3toTHDBc1h6Nrq/X7yGgpzBtpwQBchzX1aWcesIWu+XIZhEvX6ndvZ8TScj\nRM/LrLjCe8fHpLnZlQjOrOTrFzo8OUvwka80TFl66r+aILu+IUbD2S2M8GRM9RAM\nyWbAKWH4PZdHOcmHdtbh/IkMRDUK8GOhAoIBAACimf6vxrWacCI1UTyRKfivy4WV\nWvxf6U3Gzsz6i4TRiMbsjJBuOwNByu8uEzPuR/LCxU/h0v8QaXLuyd3OpcRCKkUv\n4neKFZfDwZh3WHboPDZau1zkwvoC/bdvJu1ZssWV4hx6bNU8Ddq10w4OFv+jSgQV\n2P2Sy0H3o7rFJBz3n46piOqOwRbQdu19mJvfzUQgMCp6UX9NnxgLWnFL8RTPnF01\nlwtKFRDbYghVZs+79YxKlzuxNfgfPiIxls2wV+ycOFSyAT3W5sOSusNePJSO1TCN\n3VOuLeFsjMKSPyEGhtNRHoFyZn+DXhsoFpaNh73pt+tmU+ZttumY2KvPBec=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T11:48:49.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.851+0000 7fef0423a640 1 Processor -- start 2024-09-21T11:48:49.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.852+0000 7fef0423a640 1 -- start start 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.852+0000 7fef0423a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.852+0000 7fef0423a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feefc107000 con 0x7feefc106610 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.852+0000 7fef01fd6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.852+0000 7fef01fd6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53142/0 (socket says 172.21.15.107:53142) 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.852+0000 7fef01fd6640 1 -- 172.21.15.107:0/2398635144 learned_addr learned my addr 172.21.15.107:0/2398635144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.853+0000 7fef01fd6640 1 -- 172.21.15.107:0/2398635144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc107180 con 0x7feefc106610 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.853+0000 7fef01fd6640 1 --2- 172.21.15.107:0/2398635144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc106a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feef0009820 tx=0x7feef002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef326a53f11f8de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.853+0000 7fef00fd4640 1 -- 172.21.15.107:0/2398635144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef0037070 con 0x7feefc106610 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.853+0000 7fef00fd4640 1 -- 172.21.15.107:0/2398635144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7feef0032890 con 0x7feefc106610 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.853+0000 7fef00fd4640 1 -- 172.21.15.107:0/2398635144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef0030550 con 0x7feefc106610 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 -- 172.21.15.107:0/2398635144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 msgr2=0x7feefc106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 --2- 172.21.15.107:0/2398635144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc106a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feef0009820 tx=0x7feef002a270 comp rx=0 tx=0).stop 2024-09-21T11:48:49.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 -- 172.21.15.107:0/2398635144 shutdown_connections 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 --2- 172.21.15.107:0/2398635144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc106a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 -- 172.21.15.107:0/2398635144 >> 172.21.15.107:0/2398635144 conn(0x7feefc101f00 msgr2=0x7feefc104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 -- 172.21.15.107:0/2398635144 shutdown_connections 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.854+0000 7fef0423a640 1 -- 172.21.15.107:0/2398635144 wait complete. 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef0423a640 1 Processor -- start 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef0423a640 1 -- start start 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef0423a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef0423a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10aa90 con 0x7feefc106610 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef01fd6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef01fd6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53152/0 (socket says 172.21.15.107:53152) 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef01fd6640 1 -- 172.21.15.107:0/3170480632 learned_addr learned my addr 172.21.15.107:0/3170480632 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef01fd6640 1 -- 172.21.15.107:0/3170480632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc19ab30 con 0x7feefc106610 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.855+0000 7fef01fd6640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc19a5d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feef00097f0 tx=0x7feef0030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:49.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.856+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef0037040 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.856+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7feef0040070 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.856+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feef003bd40 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.856+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc19adc0 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.856+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc19b780 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.857+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) ==== 51749+0+0 (secure 0 0 0) 0x7feef003b430 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.857+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc10ac10 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.857+0000 7feeeeffd640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7feee803f730 0x7feee8041bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.857+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feef0074320 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.857+0000 7fef017d5640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7feee803f730 0x7feee8041bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.857+0000 7fef017d5640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7feee803f730 0x7feee8041bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feef80098c0 tx=0x7feef8006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.862+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feef002e200 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:47.989+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 13) ==== 51855+0+0 (secure 0 0 0) 0x7feef0041d60 con 0x7feefc106610 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.059+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}) -- 0x7feefc103170 con 0x7feee803f730 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.779+0000 7feeeeffd640 1 -- 172.21.15.107:0/3170480632 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+5166 (secure 0 0 0) 0x7feefc103170 con 0x7feee803f730 2024-09-21T11:48:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7feee803f730 msgr2=0x7feee8041bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7feee803f730 0x7feee8041bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feef80098c0 tx=0x7feef8006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 msgr2=0x7feefc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc19a5d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feef00097f0 tx=0x7feef0030ad0 comp rx=0 tx=0).stop 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 shutdown_connections 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7feee803f730 0x7feee8041bf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 --2- 172.21.15.107:0/3170480632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc106610 0x7feefc19a5d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 >> 172.21.15.107:0/3170480632 conn(0x7feefc101f00 msgr2=0x7feefc102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 shutdown_connections 2024-09-21T11:48:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:48.782+0000 7fef0423a640 1 -- 172.21.15.107:0/3170480632 wait complete. 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f7383805640 1 Processor -- start 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f7383805640 1 -- start start 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f7383805640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f7383805640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f737c109710 con 0x7f737c07a450 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f73815a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f73815a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53162/0 (socket says 172.21.15.107:53162) 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f73815a1640 1 -- 172.21.15.107:0/2460111660 learned_addr learned my addr 172.21.15.107:0/2460111660 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.236+0000 7f73815a1640 1 -- 172.21.15.107:0/2460111660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c109890 con 0x7f737c07a450 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.237+0000 7f73815a1640 1 --2- 172.21.15.107:0/2460111660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c1091d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7378009e80 tx=0x7f737802a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a872df637987373d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.237+0000 7f7373fff640 1 -- 172.21.15.107:0/2460111660 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7378036070 con 0x7f737c07a450 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.237+0000 7f7373fff640 1 -- 172.21.15.107:0/2460111660 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7378031850 con 0x7f737c07a450 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.237+0000 7f7373fff640 1 -- 172.21.15.107:0/2460111660 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73780076d0 con 0x7f737c07a450 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 -- 172.21.15.107:0/2460111660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 msgr2=0x7f737c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 --2- 172.21.15.107:0/2460111660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c1091d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7378009e80 tx=0x7f737802a830 comp rx=0 tx=0).stop 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 -- 172.21.15.107:0/2460111660 shutdown_connections 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 --2- 172.21.15.107:0/2460111660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c1091d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 -- 172.21.15.107:0/2460111660 >> 172.21.15.107:0/2460111660 conn(0x7f737c100e00 msgr2=0x7f737c103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 -- 172.21.15.107:0/2460111660 shutdown_connections 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 -- 172.21.15.107:0/2460111660 wait complete. 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.238+0000 7f7383805640 1 Processor -- start 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f7383805640 1 -- start start 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f7383805640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:51.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f7383805640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f737c10a940 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73815a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73815a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53176/0 (socket says 172.21.15.107:53176) 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73815a1640 1 -- 172.21.15.107:0/334960655 learned_addr learned my addr 172.21.15.107:0/334960655 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73815a1640 1 -- 172.21.15.107:0/334960655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c19ab30 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73815a1640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c19a5d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7378000f80 tx=0x7f7378007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7378041020 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f737803c470 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c19adc0 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7378036040 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.239+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c19b800 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.240+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c1071c0 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.240+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 51855+0+0 (secure 0 0 0) 0x7f7378044020 con 0x7f737c07a450 2024-09-21T11:48:51.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.241+0000 7f73727fc640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f736803f7f0 0x7f7368041cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.241+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7378075430 con 0x7f737c07a450 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.241+0000 7f7380da0640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f736803f7f0 0x7f7368041cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.241+0000 7f7380da0640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f736803f7f0 0x7f7368041cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098c0 tx=0x7f736c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.245+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f737804b020 con 0x7f737c07a450 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.438+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}) -- 0x7f737c19b9d0 con 0x7f736803f7f0 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.441+0000 7f73727fc640 1 -- 172.21.15.107:0/334960655 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f737c19b9d0 con 0x7f736803f7f0 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f736803f7f0 msgr2=0x7f7368041cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f736803f7f0 0x7f7368041cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098c0 tx=0x7f736c006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 msgr2=0x7f737c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:51.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c19a5d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7378000f80 tx=0x7f7378007800 comp rx=0 tx=0).stop 2024-09-21T11:48:51.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 shutdown_connections 2024-09-21T11:48:51.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f736803f7f0 0x7f7368041cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:51.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 --2- 172.21.15.107:0/334960655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f737c07a450 0x7f737c19a5d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:51.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 >> 172.21.15.107:0/334960655 conn(0x7f737c100e00 msgr2=0x7f737c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:51.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 shutdown_connections 2024-09-21T11:48:51.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:50.444+0000 7f7383805640 1 -- 172.21.15.107:0/334960655 wait complete. 2024-09-21T11:48:51.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:51 smithi107 ceph-mon[30293]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:51.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:51 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd12b1d6640 1 Processor -- start 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd12b1d6640 1 -- start start 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd12b1d6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd124109270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd12b1d6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1241097b0 con 0x7fd12407a3b0 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd128f72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd124109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd128f72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd124109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53192/0 (socket says 172.21.15.107:53192) 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd128f72640 1 -- 172.21.15.107:0/2890078371 learned_addr learned my addr 172.21.15.107:0/2890078371 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.494+0000 7fd128f72640 1 -- 172.21.15.107:0/2890078371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd124109930 con 0x7fd12407a3b0 2024-09-21T11:48:52.328 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.495+0000 7fd128f72640 1 --2- 172.21.15.107:0/2890078371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd124109270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd118009820 tx=0x7fd11802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64265e4c1559d4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.495+0000 7fd1237fe640 1 -- 172.21.15.107:0/2890078371 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd118037070 con 0x7fd12407a3b0 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.495+0000 7fd1237fe640 1 -- 172.21.15.107:0/2890078371 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd118032890 con 0x7fd12407a3b0 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.495+0000 7fd1237fe640 1 -- 172.21.15.107:0/2890078371 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd118030550 con 0x7fd12407a3b0 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 -- 172.21.15.107:0/2890078371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 msgr2=0x7fd124109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 --2- 172.21.15.107:0/2890078371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd124109270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd118009820 tx=0x7fd11802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 -- 172.21.15.107:0/2890078371 shutdown_connections 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 --2- 172.21.15.107:0/2890078371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd124109270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 -- 172.21.15.107:0/2890078371 >> 172.21.15.107:0/2890078371 conn(0x7fd124100e00 msgr2=0x7fd124103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 -- 172.21.15.107:0/2890078371 shutdown_connections 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 -- 172.21.15.107:0/2890078371 wait complete. 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 Processor -- start 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.496+0000 7fd12b1d6640 1 -- start start 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd12b1d6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd1241068c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd12b1d6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd12410a930 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd128f72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd1241068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd128f72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd1241068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53206/0 (socket says 172.21.15.107:53206) 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd128f72640 1 -- 172.21.15.107:0/235959635 learned_addr learned my addr 172.21.15.107:0/235959635 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd128f72640 1 -- 172.21.15.107:0/235959635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd124106e20 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd128f72640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd1241068c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd118032860 tx=0x7fd118030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd118037040 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd124105210 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fd118040070 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd11803bd60 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.497+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd124105790 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.498+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 51855+0+0 (secure 0 0 0) 0x7fd118045460 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.499+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd124106ff0 con 0x7fd12407a3b0 2024-09-21T11:48:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.499+0000 7fd121ffb640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd11403f840 0x7fd114041d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.499+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd118074610 con 0x7fd12407a3b0 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.499+0000 7fd123fff640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd11403f840 0x7fd114041d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.499+0000 7fd123fff640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd11403f840 0x7fd114041d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1100098c0 tx=0x7fd110006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.503+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd118030dc0 con 0x7fd12407a3b0 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.696+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}) -- 0x7fd124107210 con 0x7fd11403f840 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.702+0000 7fd121ffb640 1 -- 172.21.15.107:0/235959635 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fd124107210 con 0x7fd11403f840 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd11403f840 msgr2=0x7fd114041d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd11403f840 0x7fd114041d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1100098c0 tx=0x7fd110006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 msgr2=0x7fd1241068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd1241068c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd118032860 tx=0x7fd118030a80 comp rx=0 tx=0).stop 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 shutdown_connections 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fd11403f840 0x7fd114041d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 --2- 172.21.15.107:0/235959635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd12407a3b0 0x7fd1241068c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:52.331 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 >> 172.21.15.107:0/235959635 conn(0x7fd124100e00 msgr2=0x7fd124101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:52.332 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 shutdown_connections 2024-09-21T11:48:52.332 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:51.705+0000 7fd12b1d6640 1 -- 172.21.15.107:0/235959635 wait complete. 2024-09-21T11:48:52.332 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-09-21T11:48:52.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:52 smithi107 ceph-mon[30293]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:52.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:52 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$rQqCHfqcYKaKMq7iv0gdLeL/msCxxkjLBuewEI6YNQrP5cylry4Uq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726919333, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb289861640 1 Processor -- start 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb289861640 1 -- start start 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb289861640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb2841058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb289861640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb284105eb0 con 0x7fb2841054c0 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb2841058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb2841058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53208/0 (socket says 172.21.15.107:53208) 2024-09-21T11:48:53.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb282ffd640 1 -- 172.21.15.107:0/1139541624 learned_addr learned my addr 172.21.15.107:0/1139541624 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.748+0000 7fb282ffd640 1 -- 172.21.15.107:0/1139541624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb284106030 con 0x7fb2841054c0 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.749+0000 7fb282ffd640 1 --2- 172.21.15.107:0/1139541624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb2841058c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb274009e80 tx=0x7fb27402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9552809ee1acefac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.749+0000 7fb281ffb640 1 -- 172.21.15.107:0/1139541624 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb274036070 con 0x7fb2841054c0 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.749+0000 7fb281ffb640 1 -- 172.21.15.107:0/1139541624 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb274031850 con 0x7fb2841054c0 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.749+0000 7fb281ffb640 1 -- 172.21.15.107:0/1139541624 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2740076d0 con 0x7fb2841054c0 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 -- 172.21.15.107:0/1139541624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 msgr2=0x7fb2841058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 --2- 172.21.15.107:0/1139541624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb2841058c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb274009e80 tx=0x7fb27402a7a0 comp rx=0 tx=0).stop 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 -- 172.21.15.107:0/1139541624 shutdown_connections 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 --2- 172.21.15.107:0/1139541624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb2841058c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 -- 172.21.15.107:0/1139541624 >> 172.21.15.107:0/1139541624 conn(0x7fb284100df0 msgr2=0x7fb284103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 -- 172.21.15.107:0/1139541624 shutdown_connections 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 -- 172.21.15.107:0/1139541624 wait complete. 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.750+0000 7fb289861640 1 Processor -- start 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb289861640 1 -- start start 2024-09-21T11:48:53.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb289861640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb28419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb289861640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb284109940 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb28419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb282ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb28419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53222/0 (socket says 172.21.15.107:53222) 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb282ffd640 1 -- 172.21.15.107:0/4239704895 learned_addr learned my addr 172.21.15.107:0/4239704895 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb282ffd640 1 -- 172.21.15.107:0/4239704895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28419ab50 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb282ffd640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb28419a5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb274009680 tx=0x7fb2740316a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb274041020 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.751+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb27403c470 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.752+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28419ade0 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.752+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb274036040 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.752+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28419b820 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.752+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 51855+0+0 (secure 0 0 0) 0x7fb274044020 con 0x7fb2841054c0 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.753+0000 7fb28885f640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb26c03f8a0 0x7fb26c041d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:53.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.753+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb284109ac0 con 0x7fb2841054c0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.753+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb274075440 con 0x7fb2841054c0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.753+0000 7fb2827fc640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb26c03f8a0 0x7fb26c041d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.753+0000 7fb2827fc640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb26c03f8a0 0x7fb26c041d60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2780098c0 tx=0x7fb278006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.757+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb27404b020 con 0x7fb2841054c0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:52.948+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb2841058c0 con 0x7fb26c03f8a0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.172+0000 7fb28885f640 1 -- 172.21.15.107:0/4239704895 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb2841058c0 con 0x7fb26c03f8a0 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb26c03f8a0 msgr2=0x7fb26c041d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb26c03f8a0 0x7fb26c041d60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb2780098c0 tx=0x7fb278006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 msgr2=0x7fb28419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb28419a5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb274009680 tx=0x7fb2740316a0 comp rx=0 tx=0).stop 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 shutdown_connections 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb26c03f8a0 0x7fb26c041d60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 --2- 172.21.15.107:0/4239704895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2841054c0 0x7fb28419a5f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:53.721 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 >> 172.21.15.107:0/4239704895 conn(0x7fb284100df0 msgr2=0x7fb284078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:53.722 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 shutdown_connections 2024-09-21T11:48:53.722 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:53.174+0000 7fb289861640 1 -- 172.21.15.107:0/4239704895 wait complete. 2024-09-21T11:48:53.722 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-09-21T11:48:54.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:54 smithi107 ceph-mon[30293]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:48:54.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:54 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.183+0000 7f5cac9c9640 1 Processor -- start 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5cac9c9640 1 -- start start 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5cac9c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca8106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5cac9c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8107000 con 0x7f5ca8106610 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5ca659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca8106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5ca659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca8106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53772/0 (socket says 172.21.15.107:53772) 2024-09-21T11:48:54.932 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5ca659c640 1 -- 172.21.15.107:0/2255917553 learned_addr learned my addr 172.21.15.107:0/2255917553 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.184+0000 7f5ca659c640 1 -- 172.21.15.107:0/2255917553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca8107180 con 0x7f5ca8106610 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.185+0000 7f5ca659c640 1 --2- 172.21.15.107:0/2255917553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca8106a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009820 tx=0x7f5c9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d14d5a231595ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.185+0000 7f5ca559a640 1 -- 172.21.15.107:0/2255917553 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c98037070 con 0x7f5ca8106610 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.185+0000 7f5ca559a640 1 -- 172.21.15.107:0/2255917553 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f5c98032890 con 0x7f5ca8106610 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.185+0000 7f5ca559a640 1 -- 172.21.15.107:0/2255917553 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c98030550 con 0x7f5ca8106610 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2255917553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 msgr2=0x7f5ca8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 --2- 172.21.15.107:0/2255917553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca8106a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009820 tx=0x7f5c9802a270 comp rx=0 tx=0).stop 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2255917553 shutdown_connections 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 --2- 172.21.15.107:0/2255917553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca8106a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2255917553 >> 172.21.15.107:0/2255917553 conn(0x7f5ca8101f00 msgr2=0x7f5ca8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2255917553 shutdown_connections 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2255917553 wait complete. 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 Processor -- start 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.186+0000 7f5cac9c9640 1 -- start start 2024-09-21T11:48:54.933 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5cac9c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5cac9c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca810aa90 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5ca659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5ca659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53784/0 (socket says 172.21.15.107:53784) 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5ca659c640 1 -- 172.21.15.107:0/2032301249 learned_addr learned my addr 172.21.15.107:0/2032301249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5ca659c640 1 -- 172.21.15.107:0/2032301249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca819ab30 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5ca659c640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca819a5d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c98032860 tx=0x7f5c98030a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c98037040 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f5c98040070 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca819adc0 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c9803bd60 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.187+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca819b780 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.188+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 51855+0+0 (secure 0 0 0) 0x7f5c98045460 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.188+0000 7f5c9f7fe640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5c9403f8a0 0x7f5c94041d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.189+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c98074490 con 0x7f5ca8106610 2024-09-21T11:48:54.934 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.189+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca810ac10 con 0x7f5ca8106610 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.189+0000 7f5ca5d9b640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5c9403f8a0 0x7f5c94041d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.189+0000 7f5ca5d9b640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5c9403f8a0 0x7f5c94041d60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c900098c0 tx=0x7f5c90006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.193+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c980443c0 con 0x7f5ca8106610 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.370+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5ca819b9e0 con 0x7f5ca8106610 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.370+0000 7f5c9f7fe640 1 -- 172.21.15.107:0/2032301249 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5c98030dc0 con 0x7f5ca8106610 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5c9403f8a0 msgr2=0x7f5c94041d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5c9403f8a0 0x7f5c94041d60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c900098c0 tx=0x7f5c90006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 msgr2=0x7f5ca819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca819a5d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c98032860 tx=0x7f5c98030a80 comp rx=0 tx=0).stop 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 shutdown_connections 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5c9403f8a0 0x7f5c94041d60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 --2- 172.21.15.107:0/2032301249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca8106610 0x7f5ca819a5d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 >> 172.21.15.107:0/2032301249 conn(0x7f5ca8101f00 msgr2=0x7f5ca8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 shutdown_connections 2024-09-21T11:48:54.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:54.373+0000 7f5cac9c9640 1 -- 172.21.15.107:0/2032301249 wait complete. 2024-09-21T11:48:54.939 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:48:54.939 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-21T11:48:54.946 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:48:54.946 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-21T11:48:54.946 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-21T11:48:54.946 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T11:48:54.948 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-09-21T11:48:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-09-21T11:48:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-09-21T11:48:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: Password: 0lntv3r1db 2024-09-21T11:48:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:54.949 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config directory 2024-09-21T11:48:55.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2032301249' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.383+0000 7fc46310b640 1 Processor -- start 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.383+0000 7fc46310b640 1 -- start start 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.384+0000 7fc46310b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.384+0000 7fc46310b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c105eb0 con 0x7fc45c1054c0 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.384+0000 7fc460ea7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.384+0000 7fc460ea7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53788/0 (socket says 172.21.15.107:53788) 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.384+0000 7fc460ea7640 1 -- 172.21.15.107:0/4167458782 learned_addr learned my addr 172.21.15.107:0/4167458782 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.384+0000 7fc460ea7640 1 -- 172.21.15.107:0/4167458782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c106030 con 0x7fc45c1054c0 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.385+0000 7fc460ea7640 1 --2- 172.21.15.107:0/4167458782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1058c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc450009820 tx=0x7fc45002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e677dfc3f8ac0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.385+0000 7fc45b7fe640 1 -- 172.21.15.107:0/4167458782 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc450037070 con 0x7fc45c1054c0 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.385+0000 7fc45b7fe640 1 -- 172.21.15.107:0/4167458782 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc450032890 con 0x7fc45c1054c0 2024-09-21T11:48:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.385+0000 7fc45b7fe640 1 -- 172.21.15.107:0/4167458782 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc450030550 con 0x7fc45c1054c0 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.385+0000 7fc46310b640 1 -- 172.21.15.107:0/4167458782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 msgr2=0x7fc45c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.385+0000 7fc46310b640 1 --2- 172.21.15.107:0/4167458782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1058c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc450009820 tx=0x7fc45002a270 comp rx=0 tx=0).stop 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 -- 172.21.15.107:0/4167458782 shutdown_connections 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 --2- 172.21.15.107:0/4167458782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1058c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 -- 172.21.15.107:0/4167458782 >> 172.21.15.107:0/4167458782 conn(0x7fc45c100df0 msgr2=0x7fc45c103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 -- 172.21.15.107:0/4167458782 shutdown_connections 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 -- 172.21.15.107:0/4167458782 wait complete. 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 Processor -- start 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 -- start start 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1903e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc46310b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c109940 con 0x7fc45c1054c0 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc460ea7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc460ea7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1903e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53804/0 (socket says 172.21.15.107:53804) 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.386+0000 7fc460ea7640 1 -- 172.21.15.107:0/898493167 learned_addr learned my addr 172.21.15.107:0/898493167 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:56.425 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc460ea7640 1 -- 172.21.15.107:0/898493167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c190940 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc460ea7640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1903e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc4500097f0 tx=0x7fc450030ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc450037040 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc450040070 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc45003bd00 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c18ed30 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.387+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c18f2b0 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.388+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 51855+0+0 (secure 0 0 0) 0x7fc45003b430 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.388+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c109ac0 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.388+0000 7fc459ffb640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc44c03f8a0 0x7fc44c041d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.388+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc4500744e0 con 0x7fc45c1054c0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.388+0000 7fc45bfff640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc44c03f8a0 0x7fc44c041d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.389+0000 7fc45bfff640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc44c03f8a0 0x7fc44c041d60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4480098c0 tx=0x7fc448006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.393+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc4500313a0 con 0x7fc45c1054c0 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.669+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fc45c18f7f0 con 0x7fc45c1054c0 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.674+0000 7fc459ffb640 1 -- 172.21.15.107:0/898493167 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fc450030c60 con 0x7fc45c1054c0 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.676+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc44c03f8a0 msgr2=0x7fc44c041d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.676+0000 7fc46310b640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc44c03f8a0 0x7fc44c041d60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4480098c0 tx=0x7fc448006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.676+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 msgr2=0x7fc45c1903e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.676+0000 7fc46310b640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1903e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc4500097f0 tx=0x7fc450030ad0 comp rx=0 tx=0).stop 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.677+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 shutdown_connections 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.677+0000 7fc46310b640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc44c03f8a0 0x7fc44c041d60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.677+0000 7fc46310b640 1 --2- 172.21.15.107:0/898493167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc45c1054c0 0x7fc45c1903e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.677+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 >> 172.21.15.107:0/898493167 conn(0x7fc45c100df0 msgr2=0x7fc45c078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.677+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 shutdown_connections 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-21T11:48:55.677+0000 7fc46310b640 1 -- 172.21.15.107:0/898493167 wait complete. 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T11:48:56.427 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:48:56.428 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-09-21T11:48:56.462 INFO:tasks.cephadm:Fetching config... 2024-09-21T11:48:56.462 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:48:56.462 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T11:48:56.477 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T11:48:56.477 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:48:56.478 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T11:48:56.533 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T11:48:56.533 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:48:56.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T11:48:56.599 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T11:48:56.599 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:48:56.599 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T11:48:56.655 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T11:48:56.655 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmi6N2/z/7fw83ritsvqKgbLCCyfCu+B+dlD2ZTYj9sIsBR3gqPQ6GHBguZNl0YORpmlI86IPqVgBV9mX14cQypwfTE8IY6P3VMKDBVyc8Gvrol5y0h1WKz8xWST9r/gqkfIO64jBP0pBNjQktL/NBgIGfohyw2satdthCDYdU7ut2L5+8p5Uq8E+W+GoeKxXOTLS76XT01eNIKejdiDM9EWRWErA2yzMz1QhteE1IIp4rv+TEJH87ydTiuvXPm3jYqIpf6Wd3u/eSeOO4E349NJhK9RNVdOvVokxWmgDXdNibmjARbPweFG74QblCjcmtREmfZKTKTTvDvr5El5qhFgH/CcG7e0DxdfyNuHb3bGRghGu7cOF+x034rUWzZYJcSv1YDNkWuzgmgo5TpcILnFSWbHNz6z9cyGKUeSo5I/zfl9gQkgTKtbpEnFx+vXjJ3h084XtGdzK/7sc7HOUR8XOO7NWxoEaVkuVolz6bmCBXmSN+Ag7Qvl1zCMl6N08= ceph-1795c04c-780f-11ef-baf6-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T11:48:56.728 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmi6N2/z/7fw83ritsvqKgbLCCyfCu+B+dlD2ZTYj9sIsBR3gqPQ6GHBguZNl0YORpmlI86IPqVgBV9mX14cQypwfTE8IY6P3VMKDBVyc8Gvrol5y0h1WKz8xWST9r/gqkfIO64jBP0pBNjQktL/NBgIGfohyw2satdthCDYdU7ut2L5+8p5Uq8E+W+GoeKxXOTLS76XT01eNIKejdiDM9EWRWErA2yzMz1QhteE1IIp4rv+TEJH87ydTiuvXPm3jYqIpf6Wd3u/eSeOO4E349NJhK9RNVdOvVokxWmgDXdNibmjARbPweFG74QblCjcmtREmfZKTKTTvDvr5El5qhFgH/CcG7e0DxdfyNuHb3bGRghGu7cOF+x034rUWzZYJcSv1YDNkWuzgmgo5TpcILnFSWbHNz6z9cyGKUeSo5I/zfl9gQkgTKtbpEnFx+vXjJ3h084XtGdzK/7sc7HOUR8XOO7NWxoEaVkuVolz6bmCBXmSN+Ag7Qvl1zCMl6N08= ceph-1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:56.739 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmi6N2/z/7fw83ritsvqKgbLCCyfCu+B+dlD2ZTYj9sIsBR3gqPQ6GHBguZNl0YORpmlI86IPqVgBV9mX14cQypwfTE8IY6P3VMKDBVyc8Gvrol5y0h1WKz8xWST9r/gqkfIO64jBP0pBNjQktL/NBgIGfohyw2satdthCDYdU7ut2L5+8p5Uq8E+W+GoeKxXOTLS76XT01eNIKejdiDM9EWRWErA2yzMz1QhteE1IIp4rv+TEJH87ydTiuvXPm3jYqIpf6Wd3u/eSeOO4E349NJhK9RNVdOvVokxWmgDXdNibmjARbPweFG74QblCjcmtREmfZKTKTTvDvr5El5qhFgH/CcG7e0DxdfyNuHb3bGRghGu7cOF+x034rUWzZYJcSv1YDNkWuzgmgo5TpcILnFSWbHNz6z9cyGKUeSo5I/zfl9gQkgTKtbpEnFx+vXjJ3h084XtGdzK/7sc7HOUR8XOO7NWxoEaVkuVolz6bmCBXmSN+Ag7Qvl1zCMl6N08= ceph-1795c04c-780f-11ef-baf6-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T11:48:56.953 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/898493167' entity='client.admin' 2024-09-21T11:48:56.991 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmi6N2/z/7fw83ritsvqKgbLCCyfCu+B+dlD2ZTYj9sIsBR3gqPQ6GHBguZNl0YORpmlI86IPqVgBV9mX14cQypwfTE8IY6P3VMKDBVyc8Gvrol5y0h1WKz8xWST9r/gqkfIO64jBP0pBNjQktL/NBgIGfohyw2satdthCDYdU7ut2L5+8p5Uq8E+W+GoeKxXOTLS76XT01eNIKejdiDM9EWRWErA2yzMz1QhteE1IIp4rv+TEJH87ydTiuvXPm3jYqIpf6Wd3u/eSeOO4E349NJhK9RNVdOvVokxWmgDXdNibmjARbPweFG74QblCjcmtREmfZKTKTTvDvr5El5qhFgH/CcG7e0DxdfyNuHb3bGRghGu7cOF+x034rUWzZYJcSv1YDNkWuzgmgo5TpcILnFSWbHNz6z9cyGKUeSo5I/zfl9gQkgTKtbpEnFx+vXjJ3h084XtGdzK/7sc7HOUR8XOO7NWxoEaVkuVolz6bmCBXmSN+Ag7Qvl1zCMl6N08= ceph-1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:48:57.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T11:48:57.254 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:57.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.907+0000 7f3f562cc640 1 -- 172.21.15.107:0/531229307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 msgr2=0x7f3f50101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:57.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.907+0000 7f3f562cc640 1 --2- 172.21.15.107:0/531229307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50101d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009880 tx=0x7f3f4002a5d0 comp rx=0 tx=0).stop 2024-09-21T11:48:57.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.907+0000 7f3f562cc640 1 -- 172.21.15.107:0/531229307 shutdown_connections 2024-09-21T11:48:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.907+0000 7f3f562cc640 1 --2- 172.21.15.107:0/531229307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50101d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.907+0000 7f3f562cc640 1 -- 172.21.15.107:0/531229307 >> 172.21.15.107:0/531229307 conn(0x7f3f500fc390 msgr2=0x7f3f500fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.908+0000 7f3f562cc640 1 -- 172.21.15.107:0/531229307 shutdown_connections 2024-09-21T11:48:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.908+0000 7f3f562cc640 1 -- 172.21.15.107:0/531229307 wait complete. 2024-09-21T11:48:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.908+0000 7f3f562cc640 1 Processor -- start 2024-09-21T11:48:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.908+0000 7f3f562cc640 1 -- start start 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f562cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f562cc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f501067f0 con 0x7f3f50101990 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53822/0 (socket says 172.21.15.107:53822) 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4ffff640 1 -- 172.21.15.107:0/2879989018 learned_addr learned my addr 172.21.15.107:0/2879989018 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4ffff640 1 -- 172.21.15.107:0/2879989018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f501013e0 con 0x7f3f50101990 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4ffff640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50100e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f40005ab0 tx=0x7f3f40005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f4003f070 con 0x7f3f50101990 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f50199ea0 con 0x7f3f50101990 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3f40003b70 con 0x7f3f50101990 2024-09-21T11:48:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.909+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f40035030 con 0x7f3f50101990 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.910+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5019a370 con 0x7f3f50101990 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.910+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f3f40042070 con 0x7f3f50101990 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.910+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50106f70 con 0x7f3f50101990 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.911+0000 7f3f4d7fa640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3f3803f8e0 0x7f3f38041da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.911+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f40075000 con 0x7f3f50101990 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.911+0000 7f3f4f7fe640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3f3803f8e0 0x7f3f38041da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:48:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.911+0000 7f3f4f7fe640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3f3803f8e0 0x7f3f38041da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098c0 tx=0x7f3f44006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:48:57.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:57.915+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f40039df0 con 0x7f3f50101990 2024-09-21T11:48:58.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.099+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3f5002e660 con 0x7f3f50101990 2024-09-21T11:48:58.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.101+0000 7f3f4d7fa640 1 -- 172.21.15.107:0/2879989018 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3f40003d20 con 0x7f3f50101990 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3f3803f8e0 msgr2=0x7f3f38041da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3f3803f8e0 0x7f3f38041da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098c0 tx=0x7f3f44006db0 comp rx=0 tx=0).stop 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 msgr2=0x7f3f50100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50100e80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f40005ab0 tx=0x7f3f40005090 comp rx=0 tx=0).stop 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 shutdown_connections 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3f3803f8e0 0x7f3f38041da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 --2- 172.21.15.107:0/2879989018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f50101990 0x7f3f50100e80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.103+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 >> 172.21.15.107:0/2879989018 conn(0x7f3f500fc390 msgr2=0x7f3f500fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.104+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 shutdown_connections 2024-09-21T11:48:58.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:48:58.104+0000 7f3f562cc640 1 -- 172.21.15.107:0/2879989018 wait complete. 2024-09-21T11:48:58.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:58 smithi107 ceph-mon[30293]: mgrmap e14: y(active, since 12s) 2024-09-21T11:48:58.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:48:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2879989018' entity='client.admin' 2024-09-21T11:48:59.412 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T11:48:59.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T11:49:00.072 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:00.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 -- 172.21.15.107:0/3359294845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 msgr2=0x7f2828102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:00.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 --2- 172.21.15.107:0/3359294845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f2828102820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2818009880 tx=0x7f281802a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:00.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 -- 172.21.15.107:0/3359294845 shutdown_connections 2024-09-21T11:49:00.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 --2- 172.21.15.107:0/3359294845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f2828102820 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:00.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 -- 172.21.15.107:0/3359294845 >> 172.21.15.107:0/3359294845 conn(0x7f28280fdd10 msgr2=0x7f2828100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:00.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 -- 172.21.15.107:0/3359294845 shutdown_connections 2024-09-21T11:49:00.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 -- 172.21.15.107:0/3359294845 wait complete. 2024-09-21T11:49:00.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.557+0000 7f282d037640 1 Processor -- start 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f282d037640 1 -- start start 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f282d037640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f28281971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f282d037640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28281078f0 con 0x7f2828102420 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f2826d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f28281971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f2826d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f28281971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53840/0 (socket says 172.21.15.107:53840) 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f2826d9d640 1 -- 172.21.15.107:0/891424653 learned_addr learned my addr 172.21.15.107:0/891424653 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f2826d9d640 1 -- 172.21.15.107:0/891424653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2828197710 con 0x7f2828102420 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.558+0000 7f2826d9d640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f28281971b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2818004dc0 tx=0x7f2818005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.559+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f281803f070 con 0x7f2828102420 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.559+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2818003a40 con 0x7f2828102420 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.559+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28281979a0 con 0x7f2828102420 2024-09-21T11:49:00.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.559+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2818035030 con 0x7f2828102420 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.559+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2828198360 con 0x7f2828102420 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.559+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f2818042070 con 0x7f2828102420 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.560+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2828108070 con 0x7f2828102420 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.560+0000 7f280ffff640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f281003f8e0 0x7f2810041da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.560+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2818074f10 con 0x7f2828102420 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.560+0000 7f282659c640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f281003f8e0 0x7f2810041da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:00.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.560+0000 7f282659c640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f281003f8e0 0x7f2810041da0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098c0 tx=0x7f281c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:00.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.564+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f281802f300 con 0x7f2828102420 2024-09-21T11:49:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.745+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f28281990a0 con 0x7f281003f8e0 2024-09-21T11:49:00.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.748+0000 7f280ffff640 1 -- 172.21.15.107:0/891424653 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f28281990a0 con 0x7f281003f8e0 2024-09-21T11:49:00.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f281003f8e0 msgr2=0x7f2810041da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:00.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f281003f8e0 0x7f2810041da0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098c0 tx=0x7f281c006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:00.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 msgr2=0x7f28281971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:00.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f28281971b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2818004dc0 tx=0x7f2818005090 comp rx=0 tx=0).stop 2024-09-21T11:49:00.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 shutdown_connections 2024-09-21T11:49:00.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f281003f8e0 0x7f2810041da0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:00.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 --2- 172.21.15.107:0/891424653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2828102420 0x7f28281971b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:00.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 >> 172.21.15.107:0/891424653 conn(0x7f28280fdd10 msgr2=0x7f28280fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:00.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 shutdown_connections 2024-09-21T11:49:00.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:00.751+0000 7f282d037640 1 -- 172.21.15.107:0/891424653 wait complete. 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"} : dispatch 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:01.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:01.465 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-21T11:49:01.465 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:49:01.465 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-21T11:49:01.484 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:49:01.484 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:49:01.542 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-21T11:49:01.542 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch host add smithi193 2024-09-21T11:49:01.781 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:02.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.181+0000 7fb64952b640 1 -- 172.21.15.107:0/2346514951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 msgr2=0x7fb6441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:02.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.181+0000 7fb64952b640 1 --2- 172.21.15.107:0/2346514951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb6441021c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb634009880 tx=0x7fb63402a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:02.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.181+0000 7fb64952b640 1 -- 172.21.15.107:0/2346514951 shutdown_connections 2024-09-21T11:49:02.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.181+0000 7fb64952b640 1 --2- 172.21.15.107:0/2346514951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb6441021c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:02.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.181+0000 7fb64952b640 1 -- 172.21.15.107:0/2346514951 >> 172.21.15.107:0/2346514951 conn(0x7fb6440fc8c0 msgr2=0x7fb6440fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.182+0000 7fb64952b640 1 -- 172.21.15.107:0/2346514951 shutdown_connections 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.182+0000 7fb64952b640 1 -- 172.21.15.107:0/2346514951 wait complete. 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.182+0000 7fb64952b640 1 Processor -- start 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.182+0000 7fb64952b640 1 -- start start 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.182+0000 7fb64952b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb644197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.182+0000 7fb64952b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb64410aec0 con 0x7fb644101de0 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb642ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb644197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb642ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb644197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53860/0 (socket says 172.21.15.107:53860) 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb642ffd640 1 -- 172.21.15.107:0/2163252825 learned_addr learned my addr 172.21.15.107:0/2163252825 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb642ffd640 1 -- 172.21.15.107:0/2163252825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb644197b50 con 0x7fb644101de0 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb642ffd640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb644197610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb634005d90 tx=0x7fb634003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb63403f070 con 0x7fb644101de0 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb63403bbf0 con 0x7fb644101de0 2024-09-21T11:49:02.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb644197e40 con 0x7fb644101de0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb634035030 con 0x7fb644101de0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.183+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64419cef0 con 0x7fb644101de0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.184+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7fb634042070 con 0x7fb644101de0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.184+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64406abf0 con 0x7fb644101de0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.184+0000 7fb63bfff640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb63003f8e0 0x7fb630041da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.184+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6340748b0 con 0x7fb644101de0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.184+0000 7fb6427fc640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb63003f8e0 0x7fb630041da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.185+0000 7fb6427fc640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb63003f8e0 0x7fb630041da0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0098c0 tx=0x7fb62c006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:02.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.189+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb63403b7a0 con 0x7fb644101de0 2024-09-21T11:49:02.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:02.374+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7fb644105c40 con 0x7fb63003f8e0 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:03.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:03 smithi107 ceph-mon[30293]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:03.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:03 smithi107 ceph-mon[30293]: Deploying cephadm binary to smithi193 2024-09-21T11:49:05.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.408+0000 7fb63bfff640 1 -- 172.21.15.107:0/2163252825 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb62c0094d0 con 0x7fb63003f8e0 2024-09-21T11:49:05.410 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb63003f8e0 msgr2=0x7fb630041da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb63003f8e0 0x7fb630041da0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0098c0 tx=0x7fb62c006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 msgr2=0x7fb644197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb644197610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb634005d90 tx=0x7fb634003d60 comp rx=0 tx=0).stop 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 shutdown_connections 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fb63003f8e0 0x7fb630041da0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 --2- 172.21.15.107:0/2163252825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb644101de0 0x7fb644197610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 >> 172.21.15.107:0/2163252825 conn(0x7fb6440fc8c0 msgr2=0x7fb644101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 shutdown_connections 2024-09-21T11:49:05.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:05.411+0000 7fb64952b640 1 -- 172.21.15.107:0/2163252825 wait complete. 2024-09-21T11:49:06.014 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T11:49:06.253 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:06.695 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:06 smithi107 ceph-mon[30293]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:06.695 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:06.695 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:06 smithi107 ceph-mon[30293]: Added host smithi193 2024-09-21T11:49:06.696 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:06.696 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:06.696 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 -- 172.21.15.107:0/1080580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 msgr2=0x7f11bc101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 --2- 172.21.15.107:0/1080580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc101a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009880 tx=0x7f11ac02a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 -- 172.21.15.107:0/1080580129 shutdown_connections 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 --2- 172.21.15.107:0/1080580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc101a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 -- 172.21.15.107:0/1080580129 >> 172.21.15.107:0/1080580129 conn(0x7f11bc0fd130 msgr2=0x7f11bc0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 -- 172.21.15.107:0/1080580129 shutdown_connections 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.693+0000 7f11c0b1d640 1 -- 172.21.15.107:0/1080580129 wait complete. 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11c0b1d640 1 Processor -- start 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11c0b1d640 1 -- start start 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11c0b1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11c0b1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc10ada0 con 0x7f11bc101680 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50560/0 (socket says 172.21.15.107:50560) 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11ba59c640 1 -- 172.21.15.107:0/2044664410 learned_addr learned my addr 172.21.15.107:0/2044664410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:49:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.694+0000 7f11ba59c640 1 -- 172.21.15.107:0/2044664410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc1968d0 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.695+0000 7f11ba59c640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc196390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005ab0 tx=0x7f11ac003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.695+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac03f070 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.695+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f11ac03bbf0 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.695+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc196bc0 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.695+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac035030 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.695+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc19bc70 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.696+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f11ac042070 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.696+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11bc102930 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.696+0000 7f11b37fe640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f11a803f940 0x7f11a8041e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.696+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f11ac074b80 con 0x7f11bc101680 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.696+0000 7f11b9d9b640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f11a803f940 0x7f11a8041e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.696+0000 7f11b9d9b640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f11a803f940 0x7f11a8041e00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f11a40098c0 tx=0x7f11a4006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.701+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11ac030e50 con 0x7f11bc101680 2024-09-21T11:49:06.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.886+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11bc1974c0 con 0x7f11a803f940 2024-09-21T11:49:06.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.888+0000 7f11b37fe640 1 -- 172.21.15.107:0/2044664410 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f11bc1974c0 con 0x7f11a803f940 2024-09-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:49:06.890 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f11a803f940 msgr2=0x7f11a8041e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f11a803f940 0x7f11a8041e00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f11a40098c0 tx=0x7f11a4006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 msgr2=0x7f11bc196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc196390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005ab0 tx=0x7f11ac003d80 comp rx=0 tx=0).stop 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 shutdown_connections 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f11a803f940 0x7f11a8041e00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 --2- 172.21.15.107:0/2044664410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11bc101680 0x7f11bc196390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 >> 172.21.15.107:0/2044664410 conn(0x7f11bc0fd130 msgr2=0x7f11bc104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 shutdown_connections 2024-09-21T11:49:06.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:06.890+0000 7f11c0b1d640 1 -- 172.21.15.107:0/2044664410 wait complete. 2024-09-21T11:49:07.548 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T11:49:07.548 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd crush tunables default 2024-09-21T11:49:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:07.821 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:07.821 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:07 smithi107 ceph-mon[30293]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.245+0000 7faf76598640 1 -- 172.21.15.107:0/2939932229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 msgr2=0x7faf70101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.245+0000 7faf76598640 1 --2- 172.21.15.107:0/2939932229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70101dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf60009880 tx=0x7faf6002a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 -- 172.21.15.107:0/2939932229 shutdown_connections 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 --2- 172.21.15.107:0/2939932229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70101dd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 -- 172.21.15.107:0/2939932229 >> 172.21.15.107:0/2939932229 conn(0x7faf70077140 msgr2=0x7faf70077550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 -- 172.21.15.107:0/2939932229 shutdown_connections 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 -- 172.21.15.107:0/2939932229 wait complete. 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 Processor -- start 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.246+0000 7faf76598640 1 -- start start 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf76598640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:08.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf76598640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf7010ada0 con 0x7faf701019f0 2024-09-21T11:49:08.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:08.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50572/0 (socket says 172.21.15.107:50572) 2024-09-21T11:49:08.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6ffff640 1 -- 172.21.15.107:0/2319850640 learned_addr learned my addr 172.21.15.107:0/2319850640 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:49:08.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6ffff640 1 -- 172.21.15.107:0/2319850640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf70197ac0 con 0x7faf701019f0 2024-09-21T11:49:08.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6ffff640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70197580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf60005d90 tx=0x7faf60003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:08.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf6003f070 con 0x7faf701019f0 2024-09-21T11:49:08.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7faf6003bbf0 con 0x7faf701019f0 2024-09-21T11:49:08.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf70197db0 con 0x7faf701019f0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.247+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf60035030 con 0x7faf701019f0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.248+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7019ce60 con 0x7faf701019f0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.249+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf701029b0 con 0x7faf701019f0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.249+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7faf60042070 con 0x7faf701019f0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.249+0000 7faf6d7fa640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faf5803f8e0 0x7faf58041da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.249+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faf60074dd0 con 0x7faf701019f0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.249+0000 7faf6f7fe640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faf5803f8e0 0x7faf58041da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:08.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.249+0000 7faf6f7fe640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faf5803f8e0 0x7faf58041da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faf640098c0 tx=0x7faf64006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:08.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.257+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf6003a3c0 con 0x7faf701019f0 2024-09-21T11:49:08.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.457+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7faf70101e50 con 0x7faf701019f0 2024-09-21T11:49:08.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.743+0000 7faf6d7fa640 1 -- 172.21.15.107:0/2319850640 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7faf60076090 con 0x7faf701019f0 2024-09-21T11:49:08.745 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-09-21T11:49:08.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.745+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faf5803f8e0 msgr2=0x7faf58041da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.745+0000 7faf76598640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faf5803f8e0 0x7faf58041da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faf640098c0 tx=0x7faf64006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 msgr2=0x7faf70197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70197580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf60005d90 tx=0x7faf60003d60 comp rx=0 tx=0).stop 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 shutdown_connections 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faf5803f8e0 0x7faf58041da0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 --2- 172.21.15.107:0/2319850640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf701019f0 0x7faf70197580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 >> 172.21.15.107:0/2319850640 conn(0x7faf70077140 msgr2=0x7faf7006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 shutdown_connections 2024-09-21T11:49:08.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:08.746+0000 7faf76598640 1 -- 172.21.15.107:0/2319850640 wait complete. 2024-09-21T11:49:09.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:08 smithi107 ceph-mon[30293]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2319850640' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T11:49:09.319 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-09-21T11:49:09.319 INFO:tasks.cephadm:Adding mon.c on smithi107 2024-09-21T11:49:09.319 INFO:tasks.cephadm:Adding mon.b on smithi193 2024-09-21T11:49:09.319 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi193:172.21.15.193=b' 2024-09-21T11:49:09.560 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T11:49:09.605 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2319850640' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:10.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 -- 172.21.15.193:0/1027560904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 msgr2=0x7f6be8101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:10.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 --2- 172.21.15.193:0/1027560904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8101a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8009880 tx=0x7f6bd802a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:10.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 -- 172.21.15.193:0/1027560904 shutdown_connections 2024-09-21T11:49:10.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 --2- 172.21.15.193:0/1027560904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8101a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 -- 172.21.15.193:0/1027560904 >> 172.21.15.193:0/1027560904 conn(0x7f6be80fd130 msgr2=0x7f6be80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 -- 172.21.15.193:0/1027560904 shutdown_connections 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.126+0000 7f6bee4c8640 1 -- 172.21.15.193:0/1027560904 wait complete. 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6bee4c8640 1 Processor -- start 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6bee4c8640 1 -- start start 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6bee4c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6bee4c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6be810ada0 con 0x7f6be8101680 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6be7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:47754/0 (socket says 172.21.15.193:47754) 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.127+0000 7f6be7fff640 1 -- 172.21.15.193:0/400149844 learned_addr learned my addr 172.21.15.193:0/400149844 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6be7fff640 1 -- 172.21.15.193:0/400149844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be81968d0 con 0x7f6be8101680 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6be7fff640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8196390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8005d90 tx=0x7f6bd8003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bd803f070 con 0x7f6be8101680 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be8196bc0 con 0x7f6be8101680 2024-09-21T11:49:10.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be819bc70 con 0x7f6be8101680 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6bd803bbf0 con 0x7f6be8101680 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.128+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bd8049060 con 0x7f6be8101680 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.129+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f6bd8042070 con 0x7f6be8101680 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.129+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be8102930 con 0x7f6be8101680 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.129+0000 7f6be5ffb640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6bd403f940 0x7f6bd4041e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.129+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6bd8074cb0 con 0x7f6be8101680 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.129+0000 7f6bdffff640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6bd403f940 0x7f6bd4041e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.130+0000 7f6bdffff640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6bd403f940 0x7f6bd4041e00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00098c0 tx=0x7f6bd0006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:10.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.134+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6bd8031de0 con 0x7f6be8101680 2024-09-21T11:49:10.326 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.324+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi193:172.21.15.193=b", "target": ["mon-mgr", ""]}) -- 0x7f6be81974c0 con 0x7f6bd403f940 2024-09-21T11:49:10.331 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.329+0000 7f6be5ffb640 1 -- 172.21.15.193:0/400149844 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6be81974c0 con 0x7f6bd403f940 2024-09-21T11:49:10.331 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-09-21T11:49:10.333 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6bd403f940 msgr2=0x7f6bd4041e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:10.333 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6bd403f940 0x7f6bd4041e00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00098c0 tx=0x7f6bd0006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:10.333 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 msgr2=0x7f6be8196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:10.333 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8196390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8005d90 tx=0x7f6bd8003d60 comp rx=0 tx=0).stop 2024-09-21T11:49:10.333 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 shutdown_connections 2024-09-21T11:49:10.334 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6bd403f940 0x7f6bd4041e00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:10.334 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 --2- 172.21.15.193:0/400149844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6be8101680 0x7f6be8196390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:10.334 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 >> 172.21.15.193:0/400149844 conn(0x7f6be80fd130 msgr2=0x7f6be8104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:10.334 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 shutdown_connections 2024-09-21T11:49:10.334 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:10.331+0000 7f6bee4c8640 1 -- 172.21.15.193:0/400149844 wait complete. 2024-09-21T11:49:10.827 DEBUG:teuthology.orchestra.run.smithi107:mon.c> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.c.service 2024-09-21T11:49:10.830 DEBUG:teuthology.orchestra.run.smithi193:mon.b> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.b.service 2024-09-21T11:49:10.832 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:49:10.832 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:11.495 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:11.902 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.882+0000 7f53e69e3640 1 -- 172.21.15.193:0/2919328986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 msgr2=0x7f53e0102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.882+0000 7f53e69e3640 1 --2- 172.21.15.193:0/2919328986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0102cf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009880 tx=0x7f53d002a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 -- 172.21.15.193:0/2919328986 shutdown_connections 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 --2- 172.21.15.193:0/2919328986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0102cf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 -- 172.21.15.193:0/2919328986 >> 172.21.15.193:0/2919328986 conn(0x7f53e00791f0 msgr2=0x7f53e007a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 -- 172.21.15.193:0/2919328986 shutdown_connections 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 -- 172.21.15.193:0/2919328986 wait complete. 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 Processor -- start 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 -- start start 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.883+0000 7f53e69e3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53e0075f30 con 0x7f53e0102910 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:11.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:47768/0 (socket says 172.21.15.193:47768) 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dffff640 1 -- 172.21.15.193:0/1081851636 learned_addr learned my addr 172.21.15.193:0/1081851636 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dffff640 1 -- 172.21.15.193:0/1081851636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e0199cc0 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dffff640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0199780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53d0005d90 tx=0x7f53d0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d003f070 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f53d003bbf0 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e0199fb0 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.884+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0035030 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.885+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e019f060 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.885+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e0103a50 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.886+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f53d0042070 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.886+0000 7f53dd7fa640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f53c803f8e0 0x7f53c8041da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.886+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53d0074c60 con 0x7f53e0102910 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.886+0000 7f53df7fe640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f53c803f8e0 0x7f53c8041da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:11.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.887+0000 7f53df7fe640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f53c803f8e0 0x7f53c8041da0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53d40098c0 tx=0x7f53d4006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:11.905 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:11.891+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53d003a3c0 con 0x7f53e0102910 2024-09-21T11:49:12.144 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.142+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f53e0102d70 con 0x7f53e0102910 2024-09-21T11:49:12.144 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.142+0000 7f53dd7fa640 1 -- 172.21.15.193:0/1081851636 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f53d003bda0 con 0x7f53e0102910 2024-09-21T11:49:12.144 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:49:12.144 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","modified":"2024-09-21T11:47:54.410883Z","created":"2024-09-21T11:47:54.410883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T11:49:12.145 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-21T11:49:12.146 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.144+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f53c803f8e0 msgr2=0x7f53c8041da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.144+0000 7f53e69e3640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f53c803f8e0 0x7f53c8041da0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53d40098c0 tx=0x7f53d4006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.144+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 msgr2=0x7f53e0199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.144+0000 7f53e69e3640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0199780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f53d0005d90 tx=0x7f53d0003d60 comp rx=0 tx=0).stop 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.145+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 shutdown_connections 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.145+0000 7f53e69e3640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f53c803f8e0 0x7f53c8041da0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.145+0000 7f53e69e3640 1 --2- 172.21.15.193:0/1081851636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53e0102910 0x7f53e0199780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.145+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 >> 172.21.15.193:0/1081851636 conn(0x7f53e00791f0 msgr2=0x7f53e0103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.145+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 shutdown_connections 2024-09-21T11:49:12.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:12.145+0000 7f53e69e3640 1 -- 172.21.15.193:0/1081851636 wait complete. 2024-09-21T11:49:12.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:12 smithi107 ceph-mon[30293]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi193:172.21.15.193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:12.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:12 smithi107 ceph-mon[30293]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi193:172.21.15.193=b;count:3 2024-09-21T11:49:12.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:12 smithi107 ceph-mon[30293]: Deploying daemon mon.b on smithi193 2024-09-21T11:49:13.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:13 smithi107 ceph-mon[30293]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:13.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/1081851636' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:14.498 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:49:14.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:49:14.752 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 -- 172.21.15.193:0/918233675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 msgr2=0x7f9b7c101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 --2- 172.21.15.193:0/918233675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c101090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009880 tx=0x7f9b6c02a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 -- 172.21.15.193:0/918233675 shutdown_connections 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 --2- 172.21.15.193:0/918233675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c101090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 -- 172.21.15.193:0/918233675 >> 172.21.15.193:0/918233675 conn(0x7f9b7c0fc9c0 msgr2=0x7f9b7c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 -- 172.21.15.193:0/918233675 shutdown_connections 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.253+0000 7f9b80b3a640 1 -- 172.21.15.193:0/918233675 wait complete. 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.254+0000 7f9b80b3a640 1 Processor -- start 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.254+0000 7f9b80b3a640 1 -- start start 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.254+0000 7f9b80b3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:15.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.254+0000 7f9b80b3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c10c0e0 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b7a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b7a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:47782/0 (socket says 172.21.15.193:47782) 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b7a59c640 1 -- 172.21.15.193:0/2775947335 learned_addr learned my addr 172.21.15.193:0/2775947335 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b7a59c640 1 -- 172.21.15.193:0/2775947335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c197f40 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b7a59c640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c197a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005d90 tx=0x7f9b6c003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c03f070 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c198230 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9b6c03bbf0 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c035030 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.255+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c19d2e0 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.256+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c06ad30 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.256+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f9b6c042070 con 0x7f9b7c100cb0 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.257+0000 7f9b737fe640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f9b6803f940 0x7f9b68041e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:15.263 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.257+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9b6c074cf0 con 0x7f9b7c100cb0 2024-09-21T11:49:15.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.257+0000 7f9b79d9b640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f9b6803f940 0x7f9b68041e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:15.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.257+0000 7f9b79d9b640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f9b6803f940 0x7f9b68041e00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b640098c0 tx=0x7f9b64006db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:15.264 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.262+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9b6c03a3c0 con 0x7f9b7c100cb0 2024-09-21T11:49:15.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:15 smithi107 ceph-mon[30293]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:15.518 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.516+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9b7c101110 con 0x7f9b7c100cb0 2024-09-21T11:49:15.519 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:49:15.519 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.517+0000 7f9b737fe640 1 -- 172.21.15.193:0/2775947335 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f9b6c042e30 con 0x7f9b7c100cb0 2024-09-21T11:49:15.519 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-21T11:49:15.519 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","modified":"2024-09-21T11:47:54.410883Z","created":"2024-09-21T11:47:54.410883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T11:49:15.521 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f9b6803f940 msgr2=0x7f9b68041e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:15.521 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f9b6803f940 0x7f9b68041e00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b640098c0 tx=0x7f9b64006db0 comp rx=0 tx=0).stop 2024-09-21T11:49:15.521 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 msgr2=0x7f9b7c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:15.521 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c197a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005d90 tx=0x7f9b6c003d60 comp rx=0 tx=0).stop 2024-09-21T11:49:15.521 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 shutdown_connections 2024-09-21T11:49:15.522 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f9b6803f940 0x7f9b68041e00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:15.522 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 --2- 172.21.15.193:0/2775947335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b7c100cb0 0x7f9b7c197a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:15.522 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.519+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 >> 172.21.15.193:0/2775947335 conn(0x7f9b7c0fc9c0 msgr2=0x7f9b7c105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:15.522 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.520+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 shutdown_connections 2024-09-21T11:49:15.522 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:15.520+0000 7f9b80b3a640 1 -- 172.21.15.193:0/2775947335 wait complete. 2024-09-21T11:49:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:15 smithi193 systemd[1]: Started Ceph mon.b for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:49:17.449 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:49:17.449 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:49:17.719 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:21.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 -- 172.21.15.193:0/2364300876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 msgr2=0x7f3dcc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:21.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 --2- 172.21.15.193:0/2364300876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3dcc0059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3de4101500 tx=0x7f3dd402bce0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 -- 172.21.15.193:0/2364300876 shutdown_connections 2024-09-21T11:49:21.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 --2- 172.21.15.193:0/2364300876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3dcc0059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 -- 172.21.15.193:0/2364300876 >> 172.21.15.193:0/2364300876 conn(0x7f3de40fd200 msgr2=0x7f3de40ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:21.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 -- 172.21.15.193:0/2364300876 shutdown_connections 2024-09-21T11:49:21.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.154+0000 7f3de8e54640 1 -- 172.21.15.193:0/2364300876 wait complete. 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.155+0000 7f3de8e54640 1 Processor -- start 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de8e54640 1 -- start start 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de8e54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3de4108410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de8e54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 0x7f3de4108d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de8e54640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3de406b5d0 con 0x7f3dcc003530 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de8e54640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3de406b450 con 0x7f3de4108950 2024-09-21T11:49:21.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3de4108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de1d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 0x7f3de4108d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3de4108410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:34434/0 (socket says 172.21.15.193:34434) 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de259c640 1 -- 172.21.15.193:0/4051747703 learned_addr learned my addr 172.21.15.193:0/4051747703 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de1d9b640 1 -- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 msgr2=0x7f3de4108d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de1d9b640 1 -- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 msgr2=0x7f3de4108d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de1d9b640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 0x7f3de4108d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de1d9b640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 0x7f3de4108d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de259c640 1 -- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 msgr2=0x7f3de4108d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de259c640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 0x7f3de4108d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.156+0000 7f3de259c640 1 -- 172.21.15.193:0/4051747703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de410a230 con 0x7f3dcc003530 2024-09-21T11:49:21.160 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.157+0000 7f3de259c640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3de4108410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c2d0 tx=0x7f3dd40055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:21.160 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.157+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3dd402ca50 con 0x7f3dcc003530 2024-09-21T11:49:21.160 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.157+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3dd402cbf0 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.157+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de41a3840 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.157+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3dd4004740 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.157+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41a3e20 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.158+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f3dd40048e0 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.158+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de4102a40 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.158+0000 7f3dd37fe640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3dc403f6d0 0x7f3dc4041b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.158+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3dd40755c0 con 0x7f3dcc003530 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.158+0000 7f3de1d9b640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3dc403f6d0 0x7f3dc4041b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:21.161 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.159+0000 7f3de1d9b640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3dc403f6d0 0x7f3dc4041b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8006fd0 tx=0x7f3dd8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:21.166 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.165+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3dd4042e90 con 0x7f3dcc003530 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: mon.a calling monitor election 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: mon.b calling monitor election 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.167 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: monmap epoch 2 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: last_changed 2024-09-21T11:49:15.849251+0000 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: min_mon_release 19 (squid) 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: election_strategy: 1 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: fsmap 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: mgrmap e14: y(active, since 35s) 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: overall HEALTH_OK 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:21.168 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:21.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: mon.a calling monitor election 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: mon.b calling monitor election 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: monmap epoch 2 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: last_changed 2024-09-21T11:49:15.849251+0000 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: min_mon_release 19 (squid) 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: election_strategy: 1 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: fsmap 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: mgrmap e14: y(active, since 35s) 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: overall HEALTH_OK 2024-09-21T11:49:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:21.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.414+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3de4191050 con 0x7f3dcc003530 2024-09-21T11:49:21.416 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.415+0000 7f3dd37fe640 1 -- 172.21.15.193:0/4051747703 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f3dd4047d60 con 0x7f3dcc003530 2024-09-21T11:49:21.417 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:49:21.417 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","modified":"2024-09-21T11:49:15.849251Z","created":"2024-09-21T11:47:54.410883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T11:49:21.417 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3dc403f6d0 msgr2=0x7f3dc4041b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3dc403f6d0 0x7f3dc4041b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8006fd0 tx=0x7f3dd8008040 comp rx=0 tx=0).stop 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 msgr2=0x7f3de4108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3de4108410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c2d0 tx=0x7f3dd40055e0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 shutdown_connections 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3dc403f6d0 0x7f3dc4041b90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3de4108950 0x7f3de4108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 --2- 172.21.15.193:0/4051747703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dcc003530 0x7f3de4108410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 >> 172.21.15.193:0/4051747703 conn(0x7f3de40fd200 msgr2=0x7f3de41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:21.420 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 shutdown_connections 2024-09-21T11:49:21.420 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:21.417+0000 7f3de8e54640 1 -- 172.21.15.193:0/4051747703 wait complete. 2024-09-21T11:49:21.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:21 smithi193 ceph-mon[33333]: Deploying daemon mon.c on smithi107 2024-09-21T11:49:21.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/4051747703' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:21.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:21 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:21 smithi107 ceph-mon[30293]: Deploying daemon mon.c on smithi107 2024-09-21T11:49:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/4051747703' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:21 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:23.013 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:49:23.014 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:49:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:22 smithi107 ceph-mon[30293]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:23.266 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:22 smithi193 ceph-mon[33333]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:23.716 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 -- 172.21.15.193:0/4048525780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81017a0 msgr2=0x7f30b8101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/4048525780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81017a0 0x7f30b8101b80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009880 tx=0x7f30a802a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 -- 172.21.15.193:0/4048525780 shutdown_connections 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/4048525780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b81017a0 0x7f30b8101b80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 -- 172.21.15.193:0/4048525780 >> 172.21.15.193:0/4048525780 conn(0x7f30b80fd230 msgr2=0x7f30b80ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 -- 172.21.15.193:0/4048525780 shutdown_connections 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.701+0000 7f30bd9b3640 1 -- 172.21.15.193:0/4048525780 wait complete. 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.702+0000 7f30bd9b3640 1 Processor -- start 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.702+0000 7f30bd9b3640 1 -- start start 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.702+0000 7f30bd9b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 0x7f30b81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.702+0000 7f30bd9b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 0x7f30b819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.702+0000 7f30bd9b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f30b810afd0 con 0x7f30b8197e30 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.702+0000 7f30bd9b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30b810ae50 con 0x7f30b81017a0 2024-09-21T11:49:23.717 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 0x7f30b819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 0x7f30b819d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:34456/0 (socket says 172.21.15.193:34456) 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 -- 172.21.15.193:0/836180573 learned_addr learned my addr 172.21.15.193:0/836180573 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b6ffd640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 0x7f30b81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b6ffd640 1 -- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 msgr2=0x7f30b81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b6ffd640 1 -- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 msgr2=0x7f30b81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b6ffd640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 0x7f30b81978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b6ffd640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 0x7f30b81978f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 -- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 msgr2=0x7f30b81978f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 0x7f30b81978f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 -- 172.21.15.193:0/836180573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b81984b0 con 0x7f30b8197e30 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30b67fc640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 0x7f30b819d1d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b4a0 tx=0x7f30ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f30ac010610 con 0x7f30b8197e30 2024-09-21T11:49:23.718 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b819d770 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f30ac010c00 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f30ac017b80 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.703+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b819dcb0 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.704+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f30ac017070 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.704+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b8102a50 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.704+0000 7f30bc9b1640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f30a003f680 0x7f30a0041b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.704+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f30ac00d040 con 0x7f30b8197e30 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.705+0000 7f30b6ffd640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f30a003f680 0x7f30a0041b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.705+0000 7f30b6ffd640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f30a003f680 0x7f30a0041b40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005d90 tx=0x7f30a80058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:23.719 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.709+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30ac01e030 con 0x7f30b8197e30 2024-09-21T11:49:23.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:23 smithi107 systemd[1]: Starting Ceph mon.c for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:49:23.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.958+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f30b8198a20 con 0x7f30b8197e30 2024-09-21T11:49:23.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.959+0000 7f30bc9b1640 1 -- 172.21.15.193:0/836180573 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f30ac017de0 con 0x7f30b8197e30 2024-09-21T11:49:23.961 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:49:23.961 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","modified":"2024-09-21T11:49:15.849251Z","created":"2024-09-21T11:47:54.410883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T11:49:23.961 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f30a003f680 msgr2=0x7f30a0041b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f30a003f680 0x7f30a0041b40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f30a8005d90 tx=0x7f30a80058b0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 msgr2=0x7f30b819d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 0x7f30b819d1d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00b4a0 tx=0x7f30ac00b960 comp rx=0 tx=0).stop 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 shutdown_connections 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f30a003f680 0x7f30a0041b40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b8197e30 0x7f30b819d1d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 --2- 172.21.15.193:0/836180573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30b81017a0 0x7f30b81978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:23.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 >> 172.21.15.193:0/836180573 conn(0x7f30b80fd230 msgr2=0x7f30b8108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:23.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 shutdown_connections 2024-09-21T11:49:23.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:23.961+0000 7f30bd9b3640 1 -- 172.21.15.193:0/836180573 wait complete. 2024-09-21T11:49:24.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:23 smithi107 podman[41059]: 2024-09-21 11:49:23.806037015 +0000 UTC m=+0.015952343 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T11:49:24.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:23 smithi107 podman[41059]: 2024-09-21 11:49:23.911408369 +0000 UTC m=+0.121323728 container create 6a100a689a79a5dad20ebfb05921809a4a79cf8d7fef3cd297c3dcf9ed94a14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T11:49:24.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 podman[41059]: 2024-09-21 11:49:24.121960745 +0000 UTC m=+0.331876066 container init 6a100a689a79a5dad20ebfb05921809a4a79cf8d7fef3cd297c3dcf9ed94a14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 podman[41059]: 2024-09-21 11:49:24.126165122 +0000 UTC m=+0.336080453 container start 6a100a689a79a5dad20ebfb05921809a4a79cf8d7fef3cd297c3dcf9ed94a14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev), process ceph-mon, pid 7 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pidfile_write: ignore empty --pid-file 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: load: jerasure load: lrc 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: RocksDB version: 7.9.2 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Git sha 0 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Compile date 2024-09-21 08:36:07 2024-09-21T11:49:24.156 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: DB SUMMARY 2024-09-21T11:49:24.157 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: DB Session ID: OQ9PDRO37H43P8W8ZK7D 2024-09-21T11:49:24.157 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: CURRENT file: CURRENT 2024-09-21T11:49:24.157 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T11:49:24.157 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T11:49:24.157 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T11:49:24.158 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-21T11:49:24.158 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.error_if_exists: 0 2024-09-21T11:49:24.158 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.create_if_missing: 0 2024-09-21T11:49:24.158 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.paranoid_checks: 1 2024-09-21T11:49:24.159 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T11:49:24.159 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T11:49:24.159 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T11:49:24.159 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.env: 0x5639439c4120 2024-09-21T11:49:24.159 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T11:49:24.159 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.info_log: 0x5639455b7700 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.statistics: (nil) 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.use_fsync: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_log_file_size: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.allow_fallocate: 1 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.use_direct_reads: 0 2024-09-21T11:49:24.160 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T11:49:24.161 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T11:49:24.161 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.db_log_dir: 2024-09-21T11:49:24.161 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.wal_dir: 2024-09-21T11:49:24.161 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T11:49:24.161 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.write_buffer_manager: 0x5639455bbb80 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T11:49:24.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.unordered_write: 0 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.row_cache: None 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.wal_filter: None 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T11:49:24.163 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.two_write_queues: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.wal_compression: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.atomic_flush: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.log_readahead_size: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T11:49:24.164 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_background_jobs: 2 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_background_compactions: -1 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_subcompactions: 1 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_open_files: -1 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T11:49:24.165 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_background_flushes: -1 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Compression algorithms supported: 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kZSTD supported: 0 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kXpressCompression supported: 0 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kBZip2Compression supported: 0 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kLZ4Compression supported: 1 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kZlibCompression supported: 1 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: kSnappyCompression supported: 1 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T11:49:24.166 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.merge_operator: 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_filter: None 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5639455b7320) 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T11:49:24.167 INFO:journalctl@ceph.mon.c.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_type: 0 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: data_block_index_type: 0 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_shortening: 1 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: checksum: 4 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: no_block_cache: 0 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache: 0x5639455ab610 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_options: 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: capacity : 536870912 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: num_shard_bits : 4 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: strict_capacity_limit : 0 2024-09-21T11:49:24.168 INFO:journalctl@ceph.mon.c.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_compressed: (nil) 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: persistent_cache: (nil) 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_size: 4096 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_size_deviation: 10 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_restart_interval: 16 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_block_restart_interval: 1 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: metadata_block_size: 4096 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: partition_filters: 0 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: use_delta_encoding: 1 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: filter_policy: bloomfilter 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: whole_key_filtering: 1 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: verify_compression: 0 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: format_version: 5 2024-09-21T11:49:24.169 INFO:journalctl@ceph.mon.c.smithi107.stdout: enable_index_compression: 1 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_align: 0 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout: max_auto_readahead_size: 262144 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout: prepopulate_block_cache: 0 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout: initial_auto_readahead_size: 8192 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression: NoCompression 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.num_levels: 7 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T11:49:24.170 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T11:49:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T11:49:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T11:49:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T11:49:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:49:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T11:49:24.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T11:49:24.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T11:49:24.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.inplace_update_support: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.bloom_locality: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.max_successive_merges: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.ttl: 2592000 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T11:49:24.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.enable_blob_files: false 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.min_blob_size: 0 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T11:49:24.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ed15c7e9-a87d-4928-9101-3778ff94a420 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726919364153285, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T11:49:24.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726919364153807, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726919364, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ed15c7e9-a87d-4928-9101-3778ff94a420", "db_session_id": "OQ9PDRO37H43P8W8ZK7D", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T11:49:24.512 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726919364153940, "job": 1, "event": "recovery_finished"} 2024-09-21T11:49:24.512 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T11:49:24.512 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T11:49:24.512 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5639455dae00 2024-09-21T11:49:24.512 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: DB pointer 0x5639456dc000 2024-09-21T11:49:24.512 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** DB Stats ** 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:49:24.513 INFO:journalctl@ceph.mon.c.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T11:49:24.514 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: Block cache BinnedLRUCache@0x5639455ab610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: using public_addrv [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: starting mon.c rank -1 at public addrs [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] at bind addrs [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:24.515 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(???) e0 preinit fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: e1 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: btime 2024-09-21T11:47:57:299923+0000 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: legacy client fscid: -1 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout: No filesystems configured 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T11:49:24.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi193:172.21.15.193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi193:172.21.15.193=b;count:3 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Deploying daemon mon.b on smithi193 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/1081851636' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:24.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.a calling monitor election 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.b calling monitor election 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.519 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: monmap epoch 2 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: last_changed 2024-09-21T11:49:15.849251+0000 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: min_mon_release 19 (squid) 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: election_strategy: 1 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: fsmap 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mgrmap e14: y(active, since 35s) 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: overall HEALTH_OK 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.520 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: Deploying daemon mon.c on smithi107 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/4051747703' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 ceph-mon[41097]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 bash[41059]: 6a100a689a79a5dad20ebfb05921809a4a79cf8d7fef3cd297c3dcf9ed94a14f 2024-09-21T11:49:24.521 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:24 smithi107 systemd[1]: Started Ceph mon.c for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:49:25.580 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:49:25.580 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:49:25.830 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:29.252 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 -- 172.21.15.193:0/2974497749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 msgr2=0x7f6e3c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:29.252 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/2974497749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e3c0059c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e54101500 tx=0x7f6e4402bc80 comp rx=0 tx=0).stop 2024-09-21T11:49:29.252 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 -- 172.21.15.193:0/2974497749 shutdown_connections 2024-09-21T11:49:29.253 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/2974497749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e3c0059c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.253 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 -- 172.21.15.193:0/2974497749 >> 172.21.15.193:0/2974497749 conn(0x7f6e540fd220 msgr2=0x7f6e540ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:29.253 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 -- 172.21.15.193:0/2974497749 shutdown_connections 2024-09-21T11:49:29.253 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.250+0000 7f6e5c61f640 1 -- 172.21.15.193:0/2974497749 wait complete. 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.251+0000 7f6e5c61f640 1 Processor -- start 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.251+0000 7f6e5c61f640 1 -- start start 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5c61f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e540fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5c61f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5c61f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e59bba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5a3bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e540fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:29.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5a3bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e540fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:45476/0 (socket says 172.21.15.193:45476) 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e59bba640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:46890/0 (socket says 172.21.15.193:46890) 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5abbc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5abbc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.193:44358/0 (socket says 172.21.15.193:44358) 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5a3bb640 1 -- 172.21.15.193:0/4153839416 learned_addr learned my addr 172.21.15.193:0/4153839416 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54108980 con 0x7f6e3c003530 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54108800 con 0x7f6e540fdc60 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6e54108b00 con 0x7f6e541a38d0 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5abbc640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 msgr2=0x7f6e541a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5abbc640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 msgr2=0x7f6e541a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5abbc640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T11:49:29.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e59bba640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 msgr2=0x7f6e540fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e59bba640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 msgr2=0x7f6e540fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e59bba640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e5abbc640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.252+0000 7f6e59bba640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5a3bb640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 msgr2=0x7f6e541a5cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5a3bb640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5a3bb640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 msgr2=0x7f6e540fe0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5a3bb640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5a3bb640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e541a6300 con 0x7f6e3c003530 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5a3bb640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e540fd720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6e44004c10 tx=0x7f6e44004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44033070 con 0x7f6e3c003530 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6e4402d8a0 con 0x7f6e3c003530 2024-09-21T11:49:29.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44038930 con 0x7f6e3c003530 2024-09-21T11:49:29.258 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e541a6590 con 0x7f6e3c003530 2024-09-21T11:49:29.258 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.253+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541a6ba0 con 0x7f6e3c003530 2024-09-21T11:49:29.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.254+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f6e44036070 con 0x7f6e3c003530 2024-09-21T11:49:29.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.255+0000 7f6e537fe640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6e4003fd70 0x7f6e40042230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:29.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.255+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6e44075b30 con 0x7f6e3c003530 2024-09-21T11:49:29.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.255+0000 7f6e59bba640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6e4003fd70 0x7f6e40042230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:29.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.255+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e54102a40 con 0x7f6e3c003530 2024-09-21T11:49:29.259 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.256+0000 7f6e59bba640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6e4003fd70 0x7f6e40042230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006fd0 tx=0x7f6e48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:29.262 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.260+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e44040340 con 0x7f6e3c003530 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: mon.a calling monitor election 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: mon.b calling monitor election 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: mon.c calling monitor election 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: monmap epoch 3 2024-09-21T11:49:29.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:29.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: last_changed 2024-09-21T11:49:24.181095+0000 2024-09-21T11:49:29.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:49:29.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: min_mon_release 19 (squid) 2024-09-21T11:49:29.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: election_strategy: 1 2024-09-21T11:49:29.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:49:29.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-09-21T11:49:29.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: 2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2024-09-21T11:49:29.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: fsmap 2024-09-21T11:49:29.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:29.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: mgrmap e14: y(active, since 43s) 2024-09-21T11:49:29.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: overall HEALTH_OK 2024-09-21T11:49:29.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: mon.a calling monitor election 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: mon.b calling monitor election 2024-09-21T11:49:29.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: mon.c calling monitor election 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-21T11:49:29.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: monmap epoch 3 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: last_changed 2024-09-21T11:49:24.181095+0000 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: min_mon_release 19 (squid) 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: election_strategy: 1 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: 2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: fsmap 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:29.510 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: mgrmap e14: y(active, since 43s) 2024-09-21T11:49:29.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: overall HEALTH_OK 2024-09-21T11:49:29.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.511 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.535 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.533+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6e54191630 con 0x7f6e3c003530 2024-09-21T11:49:29.536 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.534+0000 7f6e537fe640 1 -- 172.21.15.193:0/4153839416 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f6e4403fdb0 con 0x7f6e3c003530 2024-09-21T11:49:29.536 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:49:29.536 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","modified":"2024-09-21T11:49:24.181095Z","created":"2024-09-21T11:47:54.410883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3301","nonce":0},{"type":"v1","addr":"172.21.15.107:6790","nonce":0}]},"addr":"172.21.15.107:6790/0","public_addr":"172.21.15.107:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T11:49:29.537 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: mon.a calling monitor election 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: mon.b calling monitor election 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.537 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: mon.c calling monitor election 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: monmap epoch 3 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: fsid 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: last_changed 2024-09-21T11:49:24.181095+0000 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: created 2024-09-21T11:47:54.410883+0000 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: min_mon_release 19 (squid) 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: election_strategy: 1 2024-09-21T11:49:29.538 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: 2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: fsmap 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: mgrmap e14: y(active, since 43s) 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: overall HEALTH_OK 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.539 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6e4003fd70 msgr2=0x7f6e40042230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6e4003fd70 0x7f6e40042230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006fd0 tx=0x7f6e48008040 comp rx=0 tx=0).stop 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 msgr2=0x7f6e540fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e540fd720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6e44004c10 tx=0x7f6e44004c40 comp rx=0 tx=0).stop 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 shutdown_connections 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f6e4003fd70 0x7f6e40042230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e541a38d0 0x7f6e541a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e540fdc60 0x7f6e540fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 --2- 172.21.15.193:0/4153839416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e3c003530 0x7f6e540fd720 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:29.540 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 >> 172.21.15.193:0/4153839416 conn(0x7f6e540fd220 msgr2=0x7f6e5418e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:29.541 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 shutdown_connections 2024-09-21T11:49:29.541 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:29.537+0000 7f6e5c61f640 1 -- 172.21.15.193:0/4153839416 wait complete. 2024-09-21T11:49:30.169 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T11:49:30.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T11:49:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:30.432 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:30.432 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/4153839416' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:30.432 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:30.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/4153839416' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/4153839416' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:49:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:49:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 -- 172.21.15.107:0/3610607953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41036a0 msgr2=0x7fbea4103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/3610607953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41036a0 0x7fbea4103a80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009880 tx=0x7fbe9402a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 -- 172.21.15.107:0/3610607953 shutdown_connections 2024-09-21T11:49:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/3610607953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41036a0 0x7fbea4103a80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 -- 172.21.15.107:0/3610607953 >> 172.21.15.107:0/3610607953 conn(0x7fbea40fd0e0 msgr2=0x7fbea40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 -- 172.21.15.107:0/3610607953 shutdown_connections 2024-09-21T11:49:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.849+0000 7fbea8bf5640 1 -- 172.21.15.107:0/3610607953 wait complete. 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 Processor -- start 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 -- start start 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea41036a0 0x7fbea4101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbea4101890 0x7fbea4101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41a38f0 0x7fbea41a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbea410bf10 con 0x7fbea41a38f0 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.850+0000 7fbea8bf5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbea410bd90 con 0x7fbea41036a0 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea8bf5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fbea410c090 con 0x7fbea4101890 2024-09-21T11:49:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41a38f0 0x7fbea41a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea41036a0 0x7fbea4101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea41036a0 0x7fbea4101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:32956/0 (socket says 172.21.15.107:32956) 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbe99d9b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbea4101890 0x7fbea4101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea259c640 1 -- 172.21.15.107:0/1243931024 learned_addr learned my addr 172.21.15.107:0/1243931024 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 -- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbea4101890 msgr2=0x7fbea4101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbea4101890 0x7fbea4101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 -- 172.21.15.107:0/1243931024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea41036a0 msgr2=0x7fbea4101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea41036a0 0x7fbea4101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 -- 172.21.15.107:0/1243931024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea41a6320 con 0x7fbea41a38f0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbe99d9b640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbea4101890 0x7fbea4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea2d9d640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41a38f0 0x7fbea41a5ce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00bd50 tx=0x7fbe9c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c019070 con 0x7fbea41a38f0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea41a6610 con 0x7fbea41a38f0 2024-09-21T11:49:30.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fbe9c014890 con 0x7fbea41a38f0 2024-09-21T11:49:30.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0126a0 con 0x7fbea41a38f0 2024-09-21T11:49:30.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.851+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea41a6bd0 con 0x7fbea41a38f0 2024-09-21T11:49:30.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.852+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7fbe9c01c070 con 0x7fbea41a38f0 2024-09-21T11:49:30.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.852+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea4103b00 con 0x7fbea41a38f0 2024-09-21T11:49:30.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.853+0000 7fbe9bfff640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbe9003fd70 0x7fbe90042230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:30.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.853+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe9c053d60 con 0x7fbea41a38f0 2024-09-21T11:49:30.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.853+0000 7fbea259c640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbe9003fd70 0x7fbe90042230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:30.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.853+0000 7fbea259c640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbe9003fd70 0x7fbe90042230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005ec0 tx=0x7fbe94005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:30.861+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe9c0133a0 con 0x7fbea41a38f0 2024-09-21T11:49:31.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.064+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbea4109bd0 con 0x7fbea41a38f0 2024-09-21T11:49:31.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.064+0000 7fbe9bfff640 1 -- 172.21.15.107:0/1243931024 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fbe9c022c30 con 0x7fbea41a38f0 2024-09-21T11:49:31.066 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:31.066 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-09-21T11:49:31.066 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:49:31.066 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbe9003fd70 msgr2=0x7fbe90042230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbe9003fd70 0x7fbe90042230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005ec0 tx=0x7fbe94005a20 comp rx=0 tx=0).stop 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41a38f0 msgr2=0x7fbea41a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41a38f0 0x7fbea41a5ce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00bd50 tx=0x7fbe9c00be40 comp rx=0 tx=0).stop 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 shutdown_connections 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbe9003fd70 0x7fbe90042230 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:31.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea41a38f0 0x7fbea41a5ce0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:31.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbea4101890 0x7fbea4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:31.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 --2- 172.21.15.107:0/1243931024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbea41036a0 0x7fbea4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:31.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 >> 172.21.15.107:0/1243931024 conn(0x7fbea40fd0e0 msgr2=0x7fbea4106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:31.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 shutdown_connections 2024-09-21T11:49:31.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:31.067+0000 7fbea8bf5640 1 -- 172.21.15.107:0/1243931024 wait complete. 2024-09-21T11:49:31.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1243931024' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:31.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1243931024' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.637 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1243931024' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.638 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:31.639 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:31.639 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:31.639 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:31.662 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T11:49:31.662 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:49:31.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T11:49:31.688 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:49:31.688 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:49:31.754 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:49:31.754 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T11:49:31.782 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:49:31.782 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:49:31.848 INFO:tasks.cephadm:Adding mgr.y on smithi107 2024-09-21T11:49:31.848 INFO:tasks.cephadm:Adding mgr.x on smithi193 2024-09-21T11:49:31.848 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply mgr '2;smithi107=y;smithi193=x' 2024-09-21T11:49:32.125 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:32.520 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:32 smithi193 ceph-mon[33333]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:49:32.520 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:32 smithi193 ceph-mon[33333]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:49:32.521 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:32 smithi193 ceph-mon[33333]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:32.521 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:32 smithi193 ceph-mon[33333]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:32.521 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:32 smithi193 ceph-mon[33333]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T11:49:32.521 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:32 smithi193 ceph-mon[33333]: Reconfiguring daemon mon.a on smithi107 2024-09-21T11:49:32.524 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[41097]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:49:32.526 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[41097]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:49:32.526 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[41097]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:32.526 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[41097]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:32.526 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[41097]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T11:49:32.526 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[41097]: Reconfiguring daemon mon.a on smithi107 2024-09-21T11:49:32.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[30293]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:49:32.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[30293]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:49:32.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[30293]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:32.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[30293]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:49:32.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[30293]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T11:49:32.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:32 smithi107 ceph-mon[30293]: Reconfiguring daemon mon.a on smithi107 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 -- 172.21.15.193:0/45843324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 msgr2=0x7fc8f4101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/45843324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 0x7fc8f4101850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009880 tx=0x7fc8e402a5d0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 -- 172.21.15.193:0/45843324 shutdown_connections 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/45843324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 0x7fc8f4101850 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 -- 172.21.15.193:0/45843324 >> 172.21.15.193:0/45843324 conn(0x7fc8f40fcd20 msgr2=0x7fc8f40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 -- 172.21.15.193:0/45843324 shutdown_connections 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.600+0000 7fc8f9e78640 1 -- 172.21.15.193:0/45843324 wait complete. 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 Processor -- start 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 -- start start 2024-09-21T11:49:32.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 0x7fc8f406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 0x7fc8f419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f419ee00 0x7fc8f41a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4106920 con 0x7fc8f4101450 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41067a0 con 0x7fc8f419c4d0 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f9e78640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f4106aa0 con 0x7fc8f419ee00 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 0x7fc8f419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 0x7fc8f419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:46892/0 (socket says 172.21.15.193:46892) 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f2ffd640 1 -- 172.21.15.193:0/3867754201 learned_addr learned my addr 172.21.15.193:0/3867754201 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f37fe640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 0x7fc8f406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.601+0000 7fc8f3fff640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f419ee00 0x7fc8f41a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f2ffd640 1 -- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f419ee00 msgr2=0x7fc8f41a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:32.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f2ffd640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f419ee00 0x7fc8f41a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f2ffd640 1 -- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 msgr2=0x7fc8f406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f2ffd640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 0x7fc8f406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f2ffd640 1 -- 172.21.15.193:0/3867754201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41a1970 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f3fff640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f419ee00 0x7fc8f41a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f2ffd640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 0x7fc8f419e8c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b520 tx=0x7fc8e800b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8004230 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41a1c60 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc8e80043d0 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8003a70 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.602+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a21a0 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.603+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7fc8e800e070 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.603+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f41018d0 con 0x7fc8f419c4d0 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.603+0000 7fc8f0ff9640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc8dc03fd70 0x7fc8dc042230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:32.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.603+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc8e8053af0 con 0x7fc8f419c4d0 2024-09-21T11:49:32.657 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.603+0000 7fc8f37fe640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc8dc03fd70 0x7fc8dc042230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:32.657 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.604+0000 7fc8f37fe640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc8dc03fd70 0x7fc8dc042230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4000c00 tx=0x7fc8e40095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:32.657 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.608+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8e8014020 con 0x7fc8f419c4d0 2024-09-21T11:49:32.800 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.798+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi193=x", "target": ["mon-mgr", ""]}) -- 0x7fc8f4063590 con 0x7fc8dc03fd70 2024-09-21T11:49:32.805 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.804+0000 7fc8f0ff9640 1 -- 172.21.15.193:0/3867754201 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc8f4063590 con 0x7fc8dc03fd70 2024-09-21T11:49:32.806 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc8dc03fd70 msgr2=0x7fc8dc042230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc8dc03fd70 0x7fc8dc042230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4000c00 tx=0x7fc8e40095a0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 msgr2=0x7fc8f419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 0x7fc8f419e8c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b520 tx=0x7fc8e800b9e0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 shutdown_connections 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc8dc03fd70 0x7fc8dc042230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f419ee00 0x7fc8f41a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.809 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f419c4d0 0x7fc8f419e8c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.809 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 --2- 172.21.15.193:0/3867754201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f4101450 0x7fc8f406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:32.809 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 >> 172.21.15.193:0/3867754201 conn(0x7fc8f40fcd20 msgr2=0x7fc8f40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:32.809 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 shutdown_connections 2024-09-21T11:49:32.809 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:49:32.806+0000 7fc8f9e78640 1 -- 172.21.15.193:0/3867754201 wait complete. 2024-09-21T11:49:33.382 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:33.383 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.383 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.383 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.383 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:49:33.383 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:49:33.383 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:33.410 DEBUG:teuthology.orchestra.run.smithi193:mgr.x> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.x.service 2024-09-21T11:49:33.412 DEBUG:tasks.cephadm:set 0 configs 2024-09-21T11:49:33.413 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T11:49:33.413 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:49:33.413 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T11:49:33.428 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T11:49:33.428 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 11:49:23.637352103 +0000 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 11:43:51.635805911 +0000 2024-09-21T11:49:33.485 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 11:43:51.635805911 +0000 2024-09-21T11:49:33.486 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 11:43:51.635805911 +0000 2024-09-21T11:49:33.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:49:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:49:33.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:33.527 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T11:49:33.527 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T11:49:33.527 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000190684 s, 2.7 MB/s 2024-09-21T11:49:33.528 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T11:49:33.585 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-21T11:49:33.640 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T11:49:33.640 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:33.640 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-21T11:49:33.640 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:33.640 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:33.641 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 11:49:23.637352103 +0000 2024-09-21T11:49:33.641 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 11:43:51.637805878 +0000 2024-09-21T11:49:33.641 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 11:43:51.637805878 +0000 2024-09-21T11:49:33.641 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 11:43:51.637805878 +0000 2024-09-21T11:49:33.641 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T11:49:33.703 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T11:49:33.703 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T11:49:33.704 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000175684 s, 2.9 MB/s 2024-09-21T11:49:33.705 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T11:49:33.761 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 11:49:23.637352103 +0000 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 11:43:51.635805911 +0000 2024-09-21T11:49:33.816 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 11:43:51.635805911 +0000 2024-09-21T11:49:33.817 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 11:43:51.635805911 +0000 2024-09-21T11:49:33.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T11:49:33.879 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T11:49:33.879 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T11:49:33.879 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000171202 s, 3.0 MB/s 2024-09-21T11:49:33.880 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T11:49:33.935 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 11:49:23.638352087 +0000 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 11:43:51.636805894 +0000 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 11:43:51.636805894 +0000 2024-09-21T11:49:33.991 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 11:43:51.636805894 +0000 2024-09-21T11:49:33.991 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T11:49:34.055 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T11:49:34.055 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T11:49:34.055 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00019367 s, 2.6 MB/s 2024-09-21T11:49:34.056 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T11:49:34.111 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:49:34.111 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T11:49:34.128 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T11:49:34.128 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-21T11:49:34.184 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T11:49:34.184 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 11:49:14.838404991 +0000 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 11:43:51.132431969 +0000 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 11:43:51.132431969 +0000 2024-09-21T11:49:34.185 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 11:43:51.132431969 +0000 2024-09-21T11:49:34.185 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T11:49:34.248 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T11:49:34.248 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T11:49:34.248 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000196466 s, 2.6 MB/s 2024-09-21T11:49:34.250 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T11:49:34.306 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-21T11:49:34.361 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 11:49:14.839404975 +0000 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 11:43:51.131431895 +0000 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 11:43:51.131431895 +0000 2024-09-21T11:49:34.362 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 11:43:51.131431895 +0000 2024-09-21T11:49:34.362 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T11:49:34.425 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T11:49:34.425 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T11:49:34.425 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000198788 s, 2.6 MB/s 2024-09-21T11:49:34.426 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T11:49:34.482 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-21T11:49:34.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi193=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: Saving service mgr spec with placement smithi107=y;smithi193=x;count:2 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: Reconfiguring mgr.y (unknown last config time)... 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: Reconfiguring daemon mgr.y on smithi107 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi193=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: Saving service mgr spec with placement smithi107=y;smithi193=x;count:2 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: Reconfiguring mgr.y (unknown last config time)... 2024-09-21T11:49:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: Reconfiguring daemon mgr.y on smithi107 2024-09-21T11:49:34.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:34.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:34.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:34.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:34.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:34.537 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T11:49:34.537 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:34.537 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-21T11:49:34.537 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:34.538 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:34.538 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 11:49:14.839404975 +0000 2024-09-21T11:49:34.538 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 11:43:51.132431969 +0000 2024-09-21T11:49:34.538 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 11:43:51.132431969 +0000 2024-09-21T11:49:34.538 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 11:43:51.132431969 +0000 2024-09-21T11:49:34.538 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T11:49:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi193=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: Saving service mgr spec with placement smithi107=y;smithi193=x;count:2 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: Reconfiguring mgr.y (unknown last config time)... 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: Reconfiguring daemon mgr.y on smithi107 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:34.574 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T11:49:34.574 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T11:49:34.574 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000215757 s, 2.4 MB/s 2024-09-21T11:49:34.575 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T11:49:34.632 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-21T11:49:34.688 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T11:49:34.688 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:49:34.688 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-21T11:49:34.688 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:49:34.688 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:49:34.689 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 11:49:14.839404975 +0000 2024-09-21T11:49:34.689 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 11:43:51.130431822 +0000 2024-09-21T11:49:34.689 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 11:43:51.130431822 +0000 2024-09-21T11:49:34.689 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 11:43:51.130431822 +0000 2024-09-21T11:49:34.689 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000213219 s, 2.4 MB/s 2024-09-21T11:49:34.752 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T11:49:34.809 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-09-21T11:49:34.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T11:49:35.181 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:35 smithi193 ceph-mon[33333]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:35 smithi193 ceph-mon[33333]: Reconfiguring mon.c (monmap changed)... 2024-09-21T11:49:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:35 smithi193 ceph-mon[33333]: Reconfiguring daemon mon.c on smithi107 2024-09-21T11:49:35.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:35 smithi107 ceph-mon[30293]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:35.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:35 smithi107 ceph-mon[30293]: Reconfiguring mon.c (monmap changed)... 2024-09-21T11:49:35.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:35 smithi107 ceph-mon[30293]: Reconfiguring daemon mon.c on smithi107 2024-09-21T11:49:35.580 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:35 smithi107 ceph-mon[41097]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:35.580 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:35 smithi107 ceph-mon[41097]: Reconfiguring mon.c (monmap changed)... 2024-09-21T11:49:35.581 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:35 smithi107 ceph-mon[41097]: Reconfiguring daemon mon.c on smithi107 2024-09-21T11:49:36.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:36.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:36.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:36.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: Reconfiguring mon.b (monmap changed)... 2024-09-21T11:49:36.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:36.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:36.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:36.705 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[30293]: Reconfiguring daemon mon.b on smithi193 2024-09-21T11:49:36.705 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:36.705 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:36.705 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:36.705 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: Reconfiguring mon.b (monmap changed)... 2024-09-21T11:49:36.706 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:36.706 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:36.706 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:36.706 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:36 smithi107 ceph-mon[41097]: Reconfiguring daemon mon.b on smithi193 2024-09-21T11:49:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: Reconfiguring mon.b (monmap changed)... 2024-09-21T11:49:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:49:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:49:37.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:37.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:36 smithi193 ceph-mon[33333]: Reconfiguring daemon mon.b on smithi193 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[30293]: Deploying daemon mgr.x on smithi193 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:49:38.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T11:49:38.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:49:38.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:38.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:37 smithi107 ceph-mon[41097]: Deploying daemon mgr.x on smithi193 2024-09-21T11:49:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:38.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:37 smithi193 ceph-mon[33333]: Deploying daemon mgr.x on smithi193 2024-09-21T11:49:38.388 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:49:38.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-09-21T11:49:38.657 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:38 smithi193 ceph-mon[33333]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:39.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:38 smithi107 ceph-mon[30293]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:39.105 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:38 smithi107 ceph-mon[41097]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 -- 172.21.15.107:0/2961711794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8103420 msgr2=0x7f7be8109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2961711794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8103420 0x7f7be8109da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009d00 tx=0x7f7be002a910 comp rx=0 tx=0).stop 2024-09-21T11:49:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 -- 172.21.15.107:0/2961711794 shutdown_connections 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2961711794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8103420 0x7f7be8109da0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2961711794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be8102a60 0x7f7be8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2961711794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8101900 0x7f7be8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 -- 172.21.15.107:0/2961711794 >> 172.21.15.107:0/2961711794 conn(0x7f7be80fd1d0 msgr2=0x7f7be80ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 -- 172.21.15.107:0/2961711794 shutdown_connections 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.103+0000 7f7becc2b640 1 -- 172.21.15.107:0/2961711794 wait complete. 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 Processor -- start 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 -- start start 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8101900 0x7f7be8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8102a60 0x7f7be8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be8103420 0x7f7be81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7be810c9d0 con 0x7f7be8102a60 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7be810c850 con 0x7f7be8103420 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7becc2b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7be810cb50 con 0x7f7be8101900 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8102a60 0x7f7be8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be659c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8101900 0x7f7be8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8102a60 0x7f7be8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42112/0 (socket says 172.21.15.107:42112) 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be5d9b640 1 -- 172.21.15.107:0/2286989830 learned_addr learned my addr 172.21.15.107:0/2286989830 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be6d9d640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be8103420 0x7f7be81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be659c640 1 -- 172.21.15.107:0/2286989830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be8103420 msgr2=0x7f7be81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be659c640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be8103420 0x7f7be81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be659c640 1 -- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8102a60 msgr2=0x7f7be8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:49:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be659c640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8102a60 0x7f7be8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.104+0000 7f7be659c640 1 -- 172.21.15.107:0/2286989830 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be8199e60 con 0x7f7be8101900 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.105+0000 7f7be659c640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8101900 0x7f7be8198330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e9f0 tx=0x7f7bd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.105+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800cd80 con 0x7f7be8101900 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.105+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7bd8004470 con 0x7f7be8101900 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.105+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be819a150 con 0x7f7be8101900 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.105+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8003e10 con 0x7f7be8101900 2024-09-21T11:49:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.105+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be81a0830 con 0x7f7be8101900 2024-09-21T11:49:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.106+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 14) ==== 51899+0+0 (secure 0 0 0) 0x7f7bd8010430 con 0x7f7be8101900 2024-09-21T11:49:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.106+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be8101d00 con 0x7f7be8101900 2024-09-21T11:49:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.106+0000 7f7bd77fe640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7bd003fb20 0x7f7bd0041fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:49:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.106+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7bd8054750 con 0x7f7be8101900 2024-09-21T11:49:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.106+0000 7f7be5d9b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7bd003fb20 0x7f7bd0041fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:49:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.107+0000 7f7be5d9b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7bd003fb20 0x7f7bd0041fe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc005fd0 tx=0x7f7bdc00b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:49:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.111+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7bd8013190 con 0x7f7be8101900 2024-09-21T11:49:39.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:39.299+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7be8063590 con 0x7f7bd003fb20 2024-09-21T11:49:39.348 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mgr[35028]: -- 172.21.15.193:0/1021960431 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ca9cd70e0 con 0x562ca9cb4800 2024-09-21T11:49:39.800 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:49:39 smithi193 bash[34990]: 5edb09b6166ce23a84ed9a30796ca69f95b2f25f57022b1861413eca49dcb706 2024-09-21T11:49:39.800 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:49:39 smithi193 systemd[1]: Started Ceph mgr.x for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.193 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:40.957 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:40.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:40.959 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:40.959 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:40 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:40 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:42.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1763580383' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7e4feb96-c3c7-4f2d-9ba3-1db963213296"} : dispatch 2024-09-21T11:49:42.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1763580383' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e4feb96-c3c7-4f2d-9ba3-1db963213296"}]': finished 2024-09-21T11:49:42.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[30293]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T11:49:42.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:49:42.193 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1763580383' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7e4feb96-c3c7-4f2d-9ba3-1db963213296"} : dispatch 2024-09-21T11:49:42.193 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1763580383' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e4feb96-c3c7-4f2d-9ba3-1db963213296"}]': finished 2024-09-21T11:49:42.193 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[41097]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T11:49:42.193 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:41 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:49:42.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1763580383' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7e4feb96-c3c7-4f2d-9ba3-1db963213296"} : dispatch 2024-09-21T11:49:42.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1763580383' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e4feb96-c3c7-4f2d-9ba3-1db963213296"}]': finished 2024-09-21T11:49:42.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:41 smithi193 ceph-mon[33333]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T11:49:42.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:41 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:49:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:42 smithi107 ceph-mon[30293]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1273651150' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:49:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:42 smithi107 ceph-mon[41097]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1273651150' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:49:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:42 smithi193 ceph-mon[33333]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1273651150' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:49:44.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:49:44.948+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mgrmap(e 15) ==== 103178+0+0 (secure 0 0 0) 0x7f7bd8020cf0 con 0x7f7be8101900 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[30293]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[30293]: Standby manager daemon x started 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[41097]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[41097]: Standby manager daemon x started 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:49:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:49:45.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:49:45.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:44 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:49:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:44 smithi193 ceph-mon[33333]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:44 smithi193 ceph-mon[33333]: Standby manager daemon x started 2024-09-21T11:49:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:44 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:49:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:44 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:49:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:44 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:49:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:44 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/59190566' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:49:46.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:45 smithi107 ceph-mon[30293]: mgrmap e15: y(active, since 59s), standbys: x 2024-09-21T11:49:46.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:49:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:45 smithi107 ceph-mon[41097]: mgrmap e15: y(active, since 59s), standbys: x 2024-09-21T11:49:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:49:46.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:45 smithi193 ceph-mon[33333]: mgrmap e15: y(active, since 59s), standbys: x 2024-09-21T11:49:46.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:49:47.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:46 smithi107 ceph-mon[30293]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:46 smithi107 ceph-mon[41097]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:47.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:46 smithi193 ceph-mon[33333]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:49.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:48 smithi107 ceph-mon[30293]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:49.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:48 smithi107 ceph-mon[41097]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:48 smithi193 ceph-mon[33333]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:50.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:49 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T11:49:50.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:49 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:50.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:49 smithi107 ceph-mon[30293]: Deploying daemon osd.0 on smithi107 2024-09-21T11:49:50.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:49 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T11:49:50.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:49 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:50.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:49 smithi107 ceph-mon[41097]: Deploying daemon osd.0 on smithi107 2024-09-21T11:49:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:49 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T11:49:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:49 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:49 smithi193 ceph-mon[33333]: Deploying daemon osd.0 on smithi107 2024-09-21T11:49:51.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:50 smithi107 ceph-mon[30293]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:51.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:50 smithi107 ceph-mon[41097]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:50 smithi193 ceph-mon[33333]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:52 smithi107 ceph-mon[30293]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:52 smithi107 ceph-mon[41097]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:53.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:52 smithi193 ceph-mon[33333]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:54 smithi107 ceph-mon[30293]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:54 smithi107 ceph-mon[41097]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:54 smithi193 ceph-mon[33333]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[30293]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[41097]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:56 smithi193 ceph-mon[33333]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:59.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:58 smithi107 ceph-mon[30293]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:59.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:59.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:59.039 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:58 smithi107 ceph-mon[41097]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:59.039 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:59.039 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:59.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:58 smithi193 ceph-mon[33333]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:49:59.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:49:59.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:49:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:49:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:49:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:01.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:00 smithi107 ceph-mon[30293]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:00 smithi107 ceph-mon[30293]: overall HEALTH_OK 2024-09-21T11:50:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:00 smithi107 ceph-mon[41097]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:00 smithi107 ceph-mon[41097]: overall HEALTH_OK 2024-09-21T11:50:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:00 smithi193 ceph-mon[33333]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:00 smithi193 ceph-mon[33333]: overall HEALTH_OK 2024-09-21T11:50:02.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:01 smithi107 ceph-mon[30293]: from='osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:50:02.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:01 smithi107 ceph-mon[30293]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:50:02.245 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:01 smithi107 ceph-mon[41097]: from='osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:50:02.245 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:01 smithi107 ceph-mon[41097]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:50:02.282 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-09-21T11:50:02.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.280+0000 7f7bd77fe640 1 -- 172.21.15.107:0/2286989830 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7be8063590 con 0x7f7bd003fb20 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7bd003fb20 msgr2=0x7f7bd0041fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7bd003fb20 0x7f7bd0041fe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc005fd0 tx=0x7f7bdc00b480 comp rx=0 tx=0).stop 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8101900 msgr2=0x7f7be8198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8101900 0x7f7be8198330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e9f0 tx=0x7f7bd800eeb0 comp rx=0 tx=0).stop 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 shutdown_connections 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7bd003fb20 0x7f7bd0041fe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be8103420 0x7f7be81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7be8102a60 0x7f7be8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 --2- 172.21.15.107:0/2286989830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7be8101900 0x7f7be8198330 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 >> 172.21.15.107:0/2286989830 conn(0x7f7be80fd1d0 msgr2=0x7f7be806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 shutdown_connections 2024-09-21T11:50:02.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:02.283+0000 7f7becc2b640 1 -- 172.21.15.107:0/2286989830 wait complete. 2024-09-21T11:50:02.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:01 smithi193 ceph-mon[33333]: from='osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:50:02.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:01 smithi193 ceph-mon[33333]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:50:03.037 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.0.service 2024-09-21T11:50:03.039 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_3... 2024-09-21T11:50:03.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T11:50:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T11:50:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:03.577 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0[44809]: 2024-09-21T11:50:03.430+0000 7efee675b640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T11:50:03.577 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0[44809]: 2024-09-21T11:50:03.432+0000 7efede57a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[41097]: from='osd.0 ' entity='osd.0' 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[30293]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[30293]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T11:50:04.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:04.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:04.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:03 smithi107 ceph-mon[30293]: from='osd.0 ' entity='osd.0' 2024-09-21T11:50:04.265 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:03 smithi193 ceph-mon[33333]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:50:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:03 smithi193 ceph-mon[33333]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T11:50:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:03 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:03 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:03 smithi193 ceph-mon[33333]: from='osd.0 ' entity='osd.0' 2024-09-21T11:50:05.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:50:05.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:50:05.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:05.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] boot 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] boot 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T11:50:05.121 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:05 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:50:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:50:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:50:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: osd.0 [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] boot 2024-09-21T11:50:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T11:50:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:05 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:06.679 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:06.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T11:50:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:07 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi107 2024-09-21T11:50:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:07 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi107 to 16063M 2024-09-21T11:50:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:07 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi107 2024-09-21T11:50:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:07 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi107 to 16063M 2024-09-21T11:50:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:07 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi107 2024-09-21T11:50:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:07 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi107 to 16063M 2024-09-21T11:50:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:50:08.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-09-21T11:50:08.372 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:08.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:08 smithi107 ceph-mon[30293]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:08.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:08 smithi107 ceph-mon[41097]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:08 smithi193 ceph-mon[33333]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:08.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 -- 172.21.15.107:0/2381760274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 msgr2=0x7f7080101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:08.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 --2- 172.21.15.107:0/2381760274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f7080101d00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7070009930 tx=0x7f707002a420 comp rx=0 tx=0).stop 2024-09-21T11:50:08.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 -- 172.21.15.107:0/2381760274 shutdown_connections 2024-09-21T11:50:08.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 --2- 172.21.15.107:0/2381760274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7080103420 0x7f7080109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 --2- 172.21.15.107:0/2381760274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 0x7f7080102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 --2- 172.21.15.107:0/2381760274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f7080101d00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 -- 172.21.15.107:0/2381760274 >> 172.21.15.107:0/2381760274 conn(0x7f70800fd1d0 msgr2=0x7f70800ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.802+0000 7f7085a4f640 1 -- 172.21.15.107:0/2381760274 shutdown_connections 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 -- 172.21.15.107:0/2381760274 wait complete. 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 Processor -- start 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 -- start start 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f708006be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 0x7f708019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7080103420 0x7f708019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f708010c9d0 con 0x7f7080101900 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f708010c850 con 0x7f7080103420 2024-09-21T11:50:08.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.803+0000 7f7085a4f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f708010cb50 con 0x7f7080102a60 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f708006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f708006be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38934/0 (socket says 172.21.15.107:38934) 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707e7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 0x7f708019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 -- 172.21.15.107:0/1489212672 learned_addr learned my addr 172.21.15.107:0/1489212672 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707f7fe640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7080103420 0x7f708019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 -- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 msgr2=0x7f708019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 0x7f708019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 -- 172.21.15.107:0/1489212672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7080103420 msgr2=0x7f708019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7080103420 0x7f708019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 -- 172.21.15.107:0/1489212672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708019fef0 con 0x7f7080101900 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707e7fc640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 0x7f708019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f707effd640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f708006be00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7070009230 tx=0x7f7070004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:50:08.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707003f070 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f70700318d0 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70801a0180 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070035030 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.804+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801a06b0 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.805+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7080101d00 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.806+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 103178+0+0 (secure 0 0 0) 0x7f7070042070 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.806+0000 7f7084a4d640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f706c07afd0 0x7f706c07d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.806+0000 7f707e7fc640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f706c07afd0 0x7f706c07d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.806+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f70700bda60 con 0x7f7080101900 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.807+0000 7f707e7fc640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f706c07afd0 0x7f706c07d490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70680095f0 tx=0x7f7068005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:50:08.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.811+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f707008a2e0 con 0x7f7080101900 2024-09-21T11:50:09.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:08.998+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7080063590 con 0x7f706c07afd0 2024-09-21T11:50:09.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[30293]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:50:09.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:50:09.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:50:09.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:09.627 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[41097]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:50:09.628 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:50:09.628 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:50:09.628 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:09 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:09 smithi193 ceph-mon[33333]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:50:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:09 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:50:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:09 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:50:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:09 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:10.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:10 smithi107 ceph-mon[30293]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:10.712 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:10 smithi107 ceph-mon[41097]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:10 smithi193 ceph-mon[33333]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:11.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2293801493' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"} : dispatch 2024-09-21T11:50:11.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"} : dispatch 2024-09-21T11:50:11.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"}]': finished 2024-09-21T11:50:11.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[30293]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T11:50:11.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2293801493' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"} : dispatch 2024-09-21T11:50:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"} : dispatch 2024-09-21T11:50:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"}]': finished 2024-09-21T11:50:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[41097]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T11:50:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:11 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2293801493' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"} : dispatch 2024-09-21T11:50:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"} : dispatch 2024-09-21T11:50:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a464ccec-f1a1-48ae-b233-89519142c774"}]': finished 2024-09-21T11:50:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:11 smithi193 ceph-mon[33333]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T11:50:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:11 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:12.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:12 smithi107 ceph-mon[41097]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:12.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3963198017' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:50:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:12 smithi107 ceph-mon[30293]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3963198017' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:50:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:12 smithi193 ceph-mon[33333]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3963198017' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:50:14.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:14 smithi107 ceph-mon[30293]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:14 smithi107 ceph-mon[41097]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:14 smithi193 ceph-mon[33333]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:16.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:16 smithi107 ceph-mon[30293]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:16.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:16 smithi107 ceph-mon[41097]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:16 smithi193 ceph-mon[33333]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:18.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:18 smithi107 ceph-mon[30293]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:18.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:18 smithi107 ceph-mon[41097]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:18.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:18 smithi193 ceph-mon[33333]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:19.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:19 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T11:50:19.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:19 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:19.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:19 smithi107 ceph-mon[30293]: Deploying daemon osd.1 on smithi107 2024-09-21T11:50:19.727 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:19 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T11:50:19.727 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:19 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:19.727 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:19 smithi107 ceph-mon[41097]: Deploying daemon osd.1 on smithi107 2024-09-21T11:50:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:19 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T11:50:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:19 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:19 smithi193 ceph-mon[33333]: Deploying daemon osd.1 on smithi107 2024-09-21T11:50:20.565 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:20 smithi107 ceph-mon[30293]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:20.565 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:20 smithi107 ceph-mon[41097]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:20 smithi193 ceph-mon[33333]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:22 smithi107 ceph-mon[30293]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:22.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:22 smithi107 ceph-mon[41097]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:22 smithi193 ceph-mon[33333]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:24.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:24 smithi107 ceph-mon[30293]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:24 smithi107 ceph-mon[41097]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:24 smithi193 ceph-mon[33333]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:26.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[30293]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:26.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[41097]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:26 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:26 smithi193 ceph-mon[33333]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:26 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:26 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:26 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:28.748 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:28 smithi107 ceph-mon[30293]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:28.748 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:28 smithi107 ceph-mon[41097]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:28.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:28 smithi193 ceph-mon[33333]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:30.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:30.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:31.342 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:31 smithi107 ceph-mon[41097]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:31.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:31 smithi107 ceph-mon[30293]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:31.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:31 smithi193 ceph-mon[33333]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:31.938 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-09-21T11:50:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.936+0000 7f7084a4d640 1 -- 172.21.15.107:0/1489212672 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7080063590 con 0x7f706c07afd0 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f706c07afd0 msgr2=0x7f706c07d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f706c07afd0 0x7f706c07d490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70680095f0 tx=0x7f7068005f90 comp rx=0 tx=0).stop 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 msgr2=0x7f708006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f708006be00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7070009230 tx=0x7f7070004b80 comp rx=0 tx=0).stop 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 shutdown_connections 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f706c07afd0 0x7f706c07d490 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7080103420 0x7f708019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7080102a60 0x7f708019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 --2- 172.21.15.107:0/1489212672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7080101900 0x7f708006be00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:31.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 >> 172.21.15.107:0/1489212672 conn(0x7f70800fd1d0 msgr2=0x7f70800fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:50:31.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 shutdown_connections 2024-09-21T11:50:31.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:31.939+0000 7f7085a4f640 1 -- 172.21.15.107:0/1489212672 wait complete. 2024-09-21T11:50:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[30293]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T11:50:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[41097]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T11:50:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:32 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:32.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:32 smithi193 ceph-mon[33333]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T11:50:32.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:32 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:32.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:32 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:32.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:32 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:32.586 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.1.service 2024-09-21T11:50:32.588 INFO:tasks.cephadm:Deploying osd.2 on smithi107 with /dev/vg_nvme/lv_2... 2024-09-21T11:50:32.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[30293]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[30293]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[30293]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[30293]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[41097]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[41097]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T11:50:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[41097]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T11:50:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[41097]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:33.255 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1[50615]: 2024-09-21T11:50:33.091+0000 7f1092ad9640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T11:50:33.255 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:50:33 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1[50615]: 2024-09-21T11:50:33.094+0000 7f108e900640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:50:33.341 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:33.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:33 smithi193 ceph-mon[33333]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:33.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:33 smithi193 ceph-mon[33333]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T11:50:33.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:33 smithi193 ceph-mon[33333]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T11:50:33.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:33 smithi193 ceph-mon[33333]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:50:33.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:34.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[30293]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[30293]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[41097]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[41097]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:34.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:34 smithi193 ceph-mon[33333]: from='osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:50:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:34 smithi193 ceph-mon[33333]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T11:50:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[30293]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[30293]: osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] boot 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[30293]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[41097]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[41097]: osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] boot 2024-09-21T11:50:35.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[41097]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T11:50:35.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:35 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:35 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:50:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:35 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:50:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:35 smithi193 ceph-mon[33333]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:50:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:35 smithi193 ceph-mon[33333]: osd.1 [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] boot 2024-09-21T11:50:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:35 smithi193 ceph-mon[33333]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T11:50:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:35 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:50:36.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:36 smithi107 ceph-mon[30293]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T11:50:36.342 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:36 smithi107 ceph-mon[41097]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T11:50:36.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:36 smithi193 ceph-mon[33333]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:37.409 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:37.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:37.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:37.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:37.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:37.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:37.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:37.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:38.043 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:50:38.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-09-21T11:50:38.319 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:38.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:38 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi107 2024-09-21T11:50:38.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:38 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi107 to 8031M 2024-09-21T11:50:38.440 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:38 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi107 2024-09-21T11:50:38.440 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:38 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi107 to 8031M 2024-09-21T11:50:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:38 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi107 2024-09-21T11:50:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:38 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi107 to 8031M 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 -- 172.21.15.107:0/2398521221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c103430 msgr2=0x7faa9c109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 --2- 172.21.15.107:0/2398521221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c103430 0x7faa9c109db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa94009960 tx=0x7faa9402a460 comp rx=0 tx=0).stop 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 -- 172.21.15.107:0/2398521221 shutdown_connections 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 --2- 172.21.15.107:0/2398521221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c103430 0x7faa9c109db0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 --2- 172.21.15.107:0/2398521221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 0x7faa9c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 --2- 172.21.15.107:0/2398521221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa9c101910 0x7faa9c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 -- 172.21.15.107:0/2398521221 >> 172.21.15.107:0/2398521221 conn(0x7faa9c0fd200 msgr2=0x7faa9c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 -- 172.21.15.107:0/2398521221 shutdown_connections 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 -- 172.21.15.107:0/2398521221 wait complete. 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.742+0000 7faaa3bec640 1 Processor -- start 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 -- start start 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 0x7faa9c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 0x7faa9c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa9c103430 0x7faa9c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10ca40 con 0x7faa9c102a70 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c8c0 con 0x7faa9c103430 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa3bec640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7faa9c10cbc0 con 0x7faa9c101910 2024-09-21T11:50:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 0x7faa9c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 0x7faa9c06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:34626/0 (socket says 172.21.15.107:34626) 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 -- 172.21.15.107:0/3696030939 learned_addr learned my addr 172.21.15.107:0/3696030939 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1187640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 0x7faa9c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa2189640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa9c103430 0x7faa9c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 -- 172.21.15.107:0/3696030939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa9c103430 msgr2=0x7faa9c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa9c103430 0x7faa9c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 -- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 msgr2=0x7faa9c19d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 0x7faa9c19d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 -- 172.21.15.107:0/3696030939 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c19ff30 con 0x7faa9c101910 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1187640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 0x7faa9c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faaa1988640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 0x7faa9c06bda0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa9800eab0 tx=0x7faa9800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.743+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9800ce40 con 0x7faa9c101910 2024-09-21T11:50:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.744+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c1a0220 con 0x7faa9c101910 2024-09-21T11:50:38.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.744+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7faa980118c0 con 0x7faa9c101910 2024-09-21T11:50:38.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.744+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98010840 con 0x7faa9c101910 2024-09-21T11:50:38.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.744+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c1a0760 con 0x7faa9c101910 2024-09-21T11:50:38.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.745+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c101d10 con 0x7faa9c101910 2024-09-21T11:50:38.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.745+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 15) ==== 103178+0+0 (secure 0 0 0) 0x7faa98011a70 con 0x7faa9c101910 2024-09-21T11:50:38.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.746+0000 7faa92ffd640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faa8807afd0 0x7faa8807d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:50:38.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.746+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2199+0+0 (secure 0 0 0) 0x7faa9809d770 con 0x7faa9c101910 2024-09-21T11:50:38.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.746+0000 7faaa1187640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faa8807afd0 0x7faa8807d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:50:38.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.746+0000 7faaa1187640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faa8807afd0 0x7faa8807d490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009c30 tx=0x7faa8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:50:38.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.750+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa98016030 con 0x7faa9c101910 2024-09-21T11:50:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:50:38.938+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7faa9c063590 con 0x7faa8807afd0 2024-09-21T11:50:39.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[30293]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:39.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:50:39.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:50:39.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:39.399 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[41097]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:39.400 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:50:39.400 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:50:39.400 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:39 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:39 smithi193 ceph-mon[33333]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:50:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:50:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:39 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:40.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:40 smithi107 ceph-mon[30293]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:50:40.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:40 smithi107 ceph-mon[41097]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:50:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:40 smithi193 ceph-mon[33333]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:50:41.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:41 smithi107 ceph-mon[30293]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:41 smithi107 ceph-mon[41097]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:41 smithi193 ceph-mon[33333]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3388393222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"} : dispatch 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"} : dispatch 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"}]': finished 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[41097]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2001959188' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3388393222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"} : dispatch 2024-09-21T11:50:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"} : dispatch 2024-09-21T11:50:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"}]': finished 2024-09-21T11:50:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[30293]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T11:50:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:50:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2001959188' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:50:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3388393222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"} : dispatch 2024-09-21T11:50:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"} : dispatch 2024-09-21T11:50:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74584ac0-8833-4c1e-9f98-2bef0780286e"}]': finished 2024-09-21T11:50:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:42 smithi193 ceph-mon[33333]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T11:50:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:42 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:50:42.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2001959188' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:50:43.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:43 smithi193 ceph-mon[33333]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:43.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:43 smithi107 ceph-mon[41097]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:43 smithi107 ceph-mon[30293]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:45.498 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:45 smithi107 ceph-mon[30293]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:45.498 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:45 smithi107 ceph-mon[41097]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:45 smithi193 ceph-mon[33333]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:47.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:47 smithi107 ceph-mon[30293]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:47.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:47 smithi107 ceph-mon[41097]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:47.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:47 smithi193 ceph-mon[33333]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:49 smithi107 ceph-mon[30293]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:49 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T11:50:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:49 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:49 smithi107 ceph-mon[41097]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:49 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T11:50:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:49 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:49 smithi193 ceph-mon[33333]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:49 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T11:50:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:49 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:50.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:50 smithi107 ceph-mon[30293]: Deploying daemon osd.2 on smithi107 2024-09-21T11:50:50.370 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:50 smithi107 ceph-mon[41097]: Deploying daemon osd.2 on smithi107 2024-09-21T11:50:50.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:50 smithi193 ceph-mon[33333]: Deploying daemon osd.2 on smithi107 2024-09-21T11:50:51.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:51 smithi193 ceph-mon[33333]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:51.706 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:51 smithi107 ceph-mon[30293]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:51.707 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:51 smithi107 ceph-mon[41097]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:53 smithi193 ceph-mon[33333]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:53 smithi107 ceph-mon[30293]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:53 smithi107 ceph-mon[41097]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:55 smithi107 ceph-mon[30293]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:55 smithi107 ceph-mon[41097]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:55 smithi193 ceph-mon[33333]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:56.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[30293]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:56.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[41097]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:56 smithi193 ceph-mon[33333]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:50:58.642 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:58 smithi107 ceph-mon[30293]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:58.643 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:58 smithi107 ceph-mon[41097]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:50:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:58 smithi193 ceph-mon[33333]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:00.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:50:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:50:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:01.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:00 smithi107 ceph-mon[30293]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:00 smithi107 ceph-mon[41097]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:00 smithi193 ceph-mon[33333]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:01.585 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 2 on host 'smithi107' 2024-09-21T11:51:01.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.583+0000 7faa92ffd640 1 -- 172.21.15.107:0/3696030939 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa9c063590 con 0x7faa8807afd0 2024-09-21T11:51:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.585+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faa8807afd0 msgr2=0x7faa8807d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.585+0000 7faaa3bec640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faa8807afd0 0x7faa8807d490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009c30 tx=0x7faa8c005f90 comp rx=0 tx=0).stop 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.585+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 msgr2=0x7faa9c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.585+0000 7faaa3bec640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 0x7faa9c06bda0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa9800eab0 tx=0x7faa9800ef70 comp rx=0 tx=0).stop 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 shutdown_connections 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faa8807afd0 0x7faa8807d490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa9c103430 0x7faa9c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c102a70 0x7faa9c19d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 --2- 172.21.15.107:0/3696030939 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa9c101910 0x7faa9c06bda0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 >> 172.21.15.107:0/3696030939 conn(0x7faa9c0fd200 msgr2=0x7faa9c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 shutdown_connections 2024-09-21T11:51:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:01.586+0000 7faaa3bec640 1 -- 172.21.15.107:0/3696030939 wait complete. 2024-09-21T11:51:02.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[30293]: from='osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:51:02.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[30293]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:51:02.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:02.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:02.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:02.115 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[41097]: from='osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:51:02.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[41097]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:51:02.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:02.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:02.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:02.141 DEBUG:teuthology.orchestra.run.smithi107:osd.2> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.2.service 2024-09-21T11:51:02.143 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_1... 2024-09-21T11:51:02.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T11:51:02.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:01 smithi193 ceph-mon[33333]: from='osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:51:02.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:01 smithi193 ceph-mon[33333]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:51:02.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:02.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:02.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:02.433 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:03.203 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[41097]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:03.203 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[41097]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T11:51:03.204 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[41097]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T11:51:03.204 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:03.204 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[41097]: from='osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:03.204 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[41097]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:03.204 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[30293]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:03.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[30293]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T11:51:03.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[30293]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T11:51:03.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:03.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[30293]: from='osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:03.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-mon[30293]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:03.205 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T11:51:02.850+0000 7f3b807fb640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T11:51:03.206 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T11:51:02.853+0000 7f3b7c622640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:51:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:02 smithi193 ceph-mon[33333]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:02 smithi193 ceph-mon[33333]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T11:51:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:02 smithi193 ceph-mon[33333]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T11:51:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:02 smithi193 ceph-mon[33333]: from='osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:03.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:02 smithi193 ceph-mon[33333]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[30293]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[30293]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[41097]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[41097]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:03 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:03 smithi193 ceph-mon[33333]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:51:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:03 smithi193 ceph-mon[33333]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T11:51:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:03 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:03 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:04.857 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:51:05.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:51:05.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:05.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:05.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] boot 2024-09-21T11:51:05.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T11:51:05.198 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] boot 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T11:51:05.199 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:04 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:51:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:51:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:51:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: osd.2 [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] boot 2024-09-21T11:51:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T11:51:05.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:04 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:51:05.905 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:51:05.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-09-21T11:51:06.213 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:05 smithi107 ceph-mon[41097]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T11:51:06.213 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:05 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:51:06.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:05 smithi107 ceph-mon[30293]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T11:51:06.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:05 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:51:06.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:05 smithi193 ceph-mon[33333]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T11:51:06.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:05 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:51:06.467 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:06.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 -- 172.21.15.107:0/3397444587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 msgr2=0x7f93d810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 --2- 172.21.15.107:0/3397444587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d810a350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009900 tx=0x7f93cc02a410 comp rx=0 tx=0).stop 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 -- 172.21.15.107:0/3397444587 shutdown_connections 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 --2- 172.21.15.107:0/3397444587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93d810a910 0x7f93d810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 --2- 172.21.15.107:0/3397444587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d810a350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 --2- 172.21.15.107:0/3397444587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d806b540 0x7f93d8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 -- 172.21.15.107:0/3397444587 >> 172.21.15.107:0/3397444587 conn(0x7f93d80fd1b0 msgr2=0x7f93d80ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 -- 172.21.15.107:0/3397444587 shutdown_connections 2024-09-21T11:51:06.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.994+0000 7f93dd784640 1 -- 172.21.15.107:0/3397444587 wait complete. 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.995+0000 7f93dd784640 1 Processor -- start 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.995+0000 7f93dd784640 1 -- start start 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.995+0000 7f93dd784640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d806b540 0x7f93d819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.995+0000 7f93dd784640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.995+0000 7f93dd784640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93d810a910 0x7f93d819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93dd784640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93d810fd30 con 0x7f93d8107f60 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93dd784640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93d810fbb0 con 0x7f93d806b540 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93dd784640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f93d810feb0 con 0x7f93d810a910 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d806b540 0x7f93d819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33674/0 (socket says 172.21.15.107:33674) 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 -- 172.21.15.107:0/1167762942 learned_addr learned my addr 172.21.15.107:0/1167762942 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d77fe640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93d810a910 0x7f93d819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 -- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93d810a910 msgr2=0x7f93d819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93d810a910 0x7f93d819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 -- 172.21.15.107:0/1167762942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d806b540 msgr2=0x7f93d819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d806b540 0x7f93d819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.996+0000 7f93d67fc640 1 -- 172.21.15.107:0/1167762942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d819c060 con 0x7f93d8107f60 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.997+0000 7f93d67fc640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d819ab80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93cc005ec0 tx=0x7f93cc005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.997+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03f070 con 0x7f93d8107f60 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.997+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f93cc003a40 con 0x7f93d8107f60 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.997+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d819c2f0 con 0x7f93d8107f60 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.997+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc035030 con 0x7f93d8107f60 2024-09-21T11:51:07.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.997+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d81a2860 con 0x7f93d8107f60 2024-09-21T11:51:07.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.998+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d8107a20 con 0x7f93d8107f60 2024-09-21T11:51:07.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.999+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 103178+0+0 (secure 0 0 0) 0x7f93cc042070 con 0x7f93d8107f60 2024-09-21T11:51:07.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.999+0000 7f93bffff640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f93c007afd0 0x7f93c007d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:07.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.999+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3017+0+0 (secure 0 0 0) 0x7f93cc0bdc90 con 0x7f93d8107f60 2024-09-21T11:51:07.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:06.999+0000 7f93d6ffd640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f93c007afd0 0x7f93c007d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:07.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:07.000+0000 7f93d6ffd640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f93c007afd0 0x7f93c007d490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005d00 tx=0x7f93c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:51:07.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:07.005+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93cc0c3020 con 0x7f93d8107f60 2024-09-21T11:51:07.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[30293]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:07.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T11:51:07.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[30293]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T11:51:07.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:06 smithi107 sudo[60965]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:06 smithi107 sudo[60965]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:06 smithi107 sudo[60965]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[60965]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[60993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[60993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[60993]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[60993]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.054 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[41097]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:07.054 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T11:51:07.055 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[41097]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T11:51:07.055 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:51:07.178 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:06 smithi193 ceph-mon[33333]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:07.178 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T11:51:07.178 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:06 smithi193 ceph-mon[33333]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T11:51:07.178 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:51:07.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:07.216+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f93d8063590 con 0x7f93c007afd0 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61013]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61013]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61013]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61013]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61016]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61016]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61016]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61016]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61025]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:51:07.455 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61025]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61025]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61057]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:51:07.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61057]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61057]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61057]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61001]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61001]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61001]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61004]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61004]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.456 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61004]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61004]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61007]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61007]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61007]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61007]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61010]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61010]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61010]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.457 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 11:51:07 smithi107 sudo[61010]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35467]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:51:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35467]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35467]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35467]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35470]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:51:07.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35470]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:51:07.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35470]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:51:07.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 sudo[35470]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:07.964 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:07.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:07 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:08.244+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f93cc081860 con 0x7f93d8107f60 2024-09-21T11:51:08.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:08.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:08.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:07 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:08.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[30293]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:08.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi107 2024-09-21T11:51:08.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi107 to 5354M 2024-09-21T11:51:08.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[30293]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T11:51:08.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[30293]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-21T11:51:08.903 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[41097]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:08.903 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi107 2024-09-21T11:51:08.903 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi107 to 5354M 2024-09-21T11:51:08.903 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[41097]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T11:51:08.903 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:08 smithi107 ceph-mon[41097]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-21T11:51:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:08 smithi193 ceph-mon[33333]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:08 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi107 2024-09-21T11:51:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:08 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi107 to 5354M 2024-09-21T11:51:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:08 smithi193 ceph-mon[33333]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T11:51:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:08 smithi193 ceph-mon[33333]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[30293]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/262132968' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "661db9ba-cabf-4159-b051-379aa09acc00"} : dispatch 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/262132968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "661db9ba-cabf-4159-b051-379aa09acc00"}]': finished 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[30293]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2583115442' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:51:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[41097]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/262132968' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "661db9ba-cabf-4159-b051-379aa09acc00"} : dispatch 2024-09-21T11:51:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/262132968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "661db9ba-cabf-4159-b051-379aa09acc00"}]': finished 2024-09-21T11:51:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[41097]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T11:51:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2583115442' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:51:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:10 smithi193 ceph-mon[33333]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/262132968' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "661db9ba-cabf-4159-b051-379aa09acc00"} : dispatch 2024-09-21T11:51:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/262132968' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "661db9ba-cabf-4159-b051-379aa09acc00"}]': finished 2024-09-21T11:51:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:10 smithi193 ceph-mon[33333]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T11:51:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:10 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2583115442' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:51:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:12 smithi193 ceph-mon[33333]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:12 smithi107 ceph-mon[30293]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:12 smithi107 ceph-mon[41097]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:15.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:14 smithi107 ceph-mon[30293]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:15.202 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:14 smithi107 ceph-mon[41097]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:14 smithi193 ceph-mon[33333]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:17.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:16 smithi107 ceph-mon[30293]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:17.113 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:16 smithi107 ceph-mon[41097]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:16 smithi193 ceph-mon[33333]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:18.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:17 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T11:51:18.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:17 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:18.117 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:17 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T11:51:18.117 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:17 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:17 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T11:51:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:17 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:19.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:18 smithi107 ceph-mon[41097]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:19.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:18 smithi107 ceph-mon[41097]: Deploying daemon osd.3 on smithi107 2024-09-21T11:51:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:18 smithi107 ceph-mon[30293]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:18 smithi107 ceph-mon[30293]: Deploying daemon osd.3 on smithi107 2024-09-21T11:51:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:18 smithi193 ceph-mon[33333]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:18 smithi193 ceph-mon[33333]: Deploying daemon osd.3 on smithi107 2024-09-21T11:51:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:20 smithi107 ceph-mon[30293]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:20 smithi107 ceph-mon[41097]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:20 smithi193 ceph-mon[33333]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:23.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:22 smithi107 ceph-mon[30293]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:22 smithi107 ceph-mon[41097]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:22 smithi193 ceph-mon[33333]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:25.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:24 smithi107 ceph-mon[30293]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:25.229 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:24 smithi107 ceph-mon[41097]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:24 smithi193 ceph-mon[33333]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:26.131 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:26.131 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:26.131 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:26.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:26.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:26.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:27.153 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:26 smithi107 ceph-mon[41097]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:27.154 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:26 smithi107 ceph-mon[30293]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:26 smithi193 ceph-mon[33333]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:29.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[30293]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:29.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[41097]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:28 smithi193 ceph-mon[33333]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:30.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.728+0000 7f93bffff640 1 -- 172.21.15.107:0/1167762942 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f93d8063590 con 0x7f93c007afd0 2024-09-21T11:51:30.730 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.730+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f93c007afd0 msgr2=0x7f93c007d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.730+0000 7f93dd784640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f93c007afd0 0x7f93c007d490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005d00 tx=0x7f93c800b410 comp rx=0 tx=0).stop 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.730+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 msgr2=0x7f93d819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.730+0000 7f93dd784640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d819ab80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f93cc005ec0 tx=0x7f93cc005000 comp rx=0 tx=0).stop 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 shutdown_connections 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f93c007afd0 0x7f93c007d490 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93d810a910 0x7f93d819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107f60 0x7f93d819ab80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 --2- 172.21.15.107:0/1167762942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d806b540 0x7f93d819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 >> 172.21.15.107:0/1167762942 conn(0x7f93d80fd1b0 msgr2=0x7f93d810b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 shutdown_connections 2024-09-21T11:51:30.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:51:30.731+0000 7f93dd784640 1 -- 172.21.15.107:0/1167762942 wait complete. 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[30293]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[30293]: from='osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[30293]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[41097]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:31.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[41097]: from='osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:51:31.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[41097]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:51:31.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:31.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:31.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:30 smithi193 ceph-mon[33333]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:30 smithi193 ceph-mon[33333]: from='osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:51:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:30 smithi193 ceph-mon[33333]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:51:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:31.325 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.3.service 2024-09-21T11:51:31.328 INFO:tasks.cephadm:Deploying osd.4 on smithi193 with /dev/vg_nvme/lv_4... 2024-09-21T11:51:31.328 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T11:51:31.566 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:32.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[30293]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T11:51:32.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[30293]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T11:51:32.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[30293]: from='osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[30293]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[41097]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[41097]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[41097]: from='osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:32.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:31 smithi107 ceph-mon[41097]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:31 smithi193 ceph-mon[33333]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T11:51:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:31 smithi193 ceph-mon[33333]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T11:51:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:31 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:31 smithi193 ceph-mon[33333]: from='osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:31 smithi193 ceph-mon[33333]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T11:51:32.504 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T11:51:32.369+0000 7f47ef192640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T11:51:32.504 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T11:51:32.374+0000 7f47e6fb1640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:51:33.213 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:33.213 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:51:33.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T11:51:33.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: from='osd.3 ' entity='osd.3' 2024-09-21T11:51:33.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:32 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: from='osd.3 ' entity='osd.3' 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T11:51:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: from='osd.3 ' entity='osd.3' 2024-09-21T11:51:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:32 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:34.298 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:51:34.323 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-09-21T11:51:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:51:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] boot 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi107 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi107 to 4015M 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:34.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:34.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.565 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] boot 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi107 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi107 to 4015M 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:51:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: osd.3 [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] boot 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi107 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi107 to 4015M 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:34.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/2635591113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 msgr2=0x7f0e48101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/2635591113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e48101d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009930 tx=0x7f0e3802a420 comp rx=0 tx=0).stop 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/2635591113 shutdown_connections 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/2635591113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e48103430 0x7f0e48109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/2635591113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 0x7f0e48102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/2635591113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e48101d10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/2635591113 >> 172.21.15.193:0/2635591113 conn(0x7f0e480fd1c0 msgr2=0x7f0e480ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:51:34.990 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/2635591113 shutdown_connections 2024-09-21T11:51:34.993 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.955+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/2635591113 wait complete. 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 Processor -- start 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 -- start start 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e481984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 0x7f0e48198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e48103430 0x7f0e48199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4810cb30 con 0x7f0e48103430 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4810c9b0 con 0x7f0e48101910 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e4dc8c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0e4810ccb0 con 0x7f0e48102a70 2024-09-21T11:51:34.994 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e477fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e481984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e477fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e481984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:43056/0 (socket says 172.21.15.193:43056) 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.956+0000 7f0e477fe640 1 -- 172.21.15.193:0/3867774147 learned_addr learned my addr 172.21.15.193:0/3867774147 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e46ffd640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 0x7f0e48198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e47fff640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e48103430 0x7f0e48199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e477fe640 1 -- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 msgr2=0x7f0e48198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e477fe640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 0x7f0e48198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e477fe640 1 -- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e48103430 msgr2=0x7f0e48199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e477fe640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e48103430 0x7f0e48199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e477fe640 1 -- 172.21.15.193:0/3867774147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e48199fe0 con 0x7f0e48101910 2024-09-21T11:51:34.995 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e46ffd640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 0x7f0e48198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e477fe640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e481984b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0e3802a940 tx=0x7f0e380052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3803f070 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0e38003e50 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4819a270 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38035030 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.957+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e481a0860 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.958+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48101d90 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.959+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f0e38042070 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.959+0000 7f0e44ff9640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0e3007b060 0x7f0e3007d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.959+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3449+0+0 (secure 0 0 0) 0x7f0e380be370 con 0x7f0e48101910 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.960+0000 7f0e46ffd640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0e3007b060 0x7f0e3007d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:51:34.996 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.960+0000 7f0e46ffd640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0e3007b060 0x7f0e3007d520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009930 tx=0x7f0e3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:51:34.997 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:34.964+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e38086020 con 0x7f0e48101910 2024-09-21T11:51:35.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:35.155+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0e48063590 con 0x7f0e3007b060 2024-09-21T11:51:35.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:35 smithi193 ceph-mon[33333]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T11:51:35.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:35 smithi193 ceph-mon[33333]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:51:35.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:35 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:51:35.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:35 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:51:35.647 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:35 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[41097]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[41097]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[30293]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[30293]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:51:35.657 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:35 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:36.471 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:36 smithi193 ceph-mon[33333]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:36.669 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:36 smithi107 ceph-mon[41097]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:36.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:36 smithi107 ceph-mon[30293]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/821548646' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"} : dispatch 2024-09-21T11:51:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"} : dispatch 2024-09-21T11:51:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"}]': finished 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/1304198864' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/821548646' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"} : dispatch 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"} : dispatch 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"}]': finished 2024-09-21T11:51:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T11:51:38.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:38.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/1304198864' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:51:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/821548646' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"} : dispatch 2024-09-21T11:51:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"} : dispatch 2024-09-21T11:51:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c71418f-c007-4428-b338-3ebfc82f779d"}]': finished 2024-09-21T11:51:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T11:51:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/1304198864' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:51:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:40 smithi107 ceph-mon[41097]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:40 smithi107 ceph-mon[30293]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:40 smithi193 ceph-mon[33333]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:42 smithi193 ceph-mon[33333]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:42.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:42 smithi107 ceph-mon[41097]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:42 smithi107 ceph-mon[30293]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:44.674 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:44 smithi193 ceph-mon[33333]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:44 smithi107 ceph-mon[41097]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:44 smithi107 ceph-mon[30293]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T11:51:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:45 smithi107 ceph-mon[41097]: Deploying daemon osd.4 on smithi193 2024-09-21T11:51:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T11:51:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:45 smithi107 ceph-mon[30293]: Deploying daemon osd.4 on smithi193 2024-09-21T11:51:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T11:51:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:45 smithi193 ceph-mon[33333]: Deploying daemon osd.4 on smithi193 2024-09-21T11:51:46.451 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:46 smithi193 ceph-mon[33333]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:46.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:46 smithi107 ceph-mon[30293]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:46 smithi107 ceph-mon[41097]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:48.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:48 smithi107 ceph-mon[41097]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:48 smithi107 ceph-mon[30293]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:48 smithi193 ceph-mon[33333]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:50.702 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:50 smithi193 ceph-mon[33333]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:50.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:50 smithi107 ceph-mon[30293]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:50 smithi107 ceph-mon[41097]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:52.744 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:52 smithi193 ceph-mon[33333]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:52.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:52 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:52.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:52 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:52.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:52 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[30293]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[41097]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:52 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[30293]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[41097]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:55.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:55.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:55.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:55 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:55 smithi193 ceph-mon[33333]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:55 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:55 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:55 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:55.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:55 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:55.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:55 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:57 smithi193 ceph-mon[33333]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:57 smithi193 ceph-mon[33333]: from='osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:51:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:57 smithi193 ceph-mon[33333]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:51:57.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:57 smithi107 ceph-mon[30293]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:57 smithi107 ceph-mon[30293]: from='osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:51:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:57 smithi107 ceph-mon[30293]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:51:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:57 smithi107 ceph-mon[41097]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:57 smithi107 ceph-mon[41097]: from='osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:51:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:57 smithi107 ceph-mon[41097]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:51:57.644 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.641+0000 7f0e44ff9640 1 -- 172.21.15.193:0/3867774147 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e48063590 con 0x7f0e3007b060 2024-09-21T11:51:57.644 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 4 on host 'smithi193' 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0e3007b060 msgr2=0x7f0e3007d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0e3007b060 0x7f0e3007d520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009930 tx=0x7f0e3c008040 comp rx=0 tx=0).stop 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 msgr2=0x7f0e481984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e481984b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0e3802a940 tx=0x7f0e380052b0 comp rx=0 tx=0).stop 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 shutdown_connections 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0e3007b060 0x7f0e3007d520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e48103430 0x7f0e48199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e48102a70 0x7f0e48198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 --2- 172.21.15.193:0/3867774147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e48101910 0x7f0e481984b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:51:57.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 >> 172.21.15.193:0/3867774147 conn(0x7f0e480fd1c0 msgr2=0x7f0e4806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:51:57.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.644+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 shutdown_connections 2024-09-21T11:51:57.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:51:57.645+0000 7f0e4dc8c640 1 -- 172.21.15.193:0/3867774147 wait complete. 2024-09-21T11:51:58.267 DEBUG:teuthology.orchestra.run.smithi193:osd.4> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.4.service 2024-09-21T11:51:58.270 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-09-21T11:51:58.270 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T11:51:58.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:51:58.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:51:58.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:58.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:58.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T11:51:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T11:51:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:51:58.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:51:58.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:58.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:58.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:51:59.047 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:59.050 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T11:51:58.672+0000 7f7985bc7640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T11:51:59.050 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 11:51:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T11:51:58.675+0000 7f797d9e6640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:51:59.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:59.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:51:59.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T11:51:59.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.412 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: from='osd.4 ' entity='osd.4' 2024-09-21T11:51:59.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] boot 2024-09-21T11:51:59.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T11:51:59.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:51:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: from='osd.4 ' entity='osd.4' 2024-09-21T11:51:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] boot 2024-09-21T11:51:59.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T11:51:59.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-21T11:51:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:51:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T11:51:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:51:59.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: from='osd.4 ' entity='osd.4' 2024-09-21T11:51:59.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: osd.4 [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] boot 2024-09-21T11:51:59.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T11:51:59.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:51:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:52:00.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:00 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:52:00.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:00 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:52:00.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:00 smithi193 ceph-mon[33333]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T11:52:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:00 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:52:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:00 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:52:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:00 smithi107 ceph-mon[30293]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T11:52:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:00 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:52:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:00 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:52:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:00 smithi107 ceph-mon[41097]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T11:52:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:01 smithi193 ceph-mon[33333]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:01 smithi193 ceph-mon[33333]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T11:52:01.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:01 smithi107 ceph-mon[30293]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:01.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:01 smithi107 ceph-mon[30293]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T11:52:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:01 smithi107 ceph-mon[41097]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:01 smithi107 ceph-mon[41097]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi193 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi193 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi193 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:03.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:03.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:03.385 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:52:03.410 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-09-21T11:52:03.653 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:04.127 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 -- 172.21.15.193:0/1012458134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 msgr2=0x7fec78101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:04.127 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 --2- 172.21.15.193:0/1012458134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78101ce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fec68009930 tx=0x7fec6802a420 comp rx=0 tx=0).stop 2024-09-21T11:52:04.127 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 -- 172.21.15.193:0/1012458134 shutdown_connections 2024-09-21T11:52:04.127 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 --2- 172.21.15.193:0/1012458134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec78103400 0x7fec78109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 --2- 172.21.15.193:0/1012458134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec78102a40 0x7fec78102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 --2- 172.21.15.193:0/1012458134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78101ce0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 -- 172.21.15.193:0/1012458134 >> 172.21.15.193:0/1012458134 conn(0x7fec780fd1b0 msgr2=0x7fec780ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 -- 172.21.15.193:0/1012458134 shutdown_connections 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 -- 172.21.15.193:0/1012458134 wait complete. 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 Processor -- start 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.101+0000 7fec7d593640 1 -- start start 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:04.128 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec78102a40 0x7fec781989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec78103400 0x7fec781995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810db50 con 0x7fec78102a40 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810d9d0 con 0x7fec781018e0 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fec7810dcd0 con 0x7fec78103400 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39074/0 (socket says 172.21.15.193:39074) 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 -- 172.21.15.193:0/2230895099 learned_addr learned my addr 172.21.15.193:0/2230895099 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec777fe640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec78103400 0x7fec781995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec6e7fc640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec78102a40 0x7fec781989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 -- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec78103400 msgr2=0x7fec781995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec78103400 0x7fec781995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:04.129 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 -- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec78102a40 msgr2=0x7fec781989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec78102a40 0x7fec781989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 -- 172.21.15.193:0/2230895099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec78199e90 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec76ffd640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78198360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fec68005940 tx=0x7fec68004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6803f070 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fec68005410 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7819a120 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68035030 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.102+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781a0690 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.103+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec78101ce0 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.104+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7fec68042070 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.105+0000 7fec74ff9640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fec6407b060 0x7fec6407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.105+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 3991+0+0 (secure 0 0 0) 0x7fec680be500 con 0x7fec781018e0 2024-09-21T11:52:04.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.105+0000 7fec6e7fc640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fec6407b060 0x7fec6407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:04.131 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.105+0000 7fec6e7fc640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fec6407b060 0x7fec6407d520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec600058c0 tx=0x7fec6000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:52:04.131 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.109+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec68085f90 con 0x7fec781018e0 2024-09-21T11:52:04.300 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:04.297+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fec78063590 con 0x7fec6407b060 2024-09-21T11:52:05.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[30293]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[30293]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[41097]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[41097]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:52:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:04 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:05.005 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:04 smithi193 ceph-mon[33333]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:05.005 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:04 smithi193 ceph-mon[33333]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:52:05.005 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:04 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:52:05.005 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:04 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:52:05.005 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:04 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:06 smithi193 ceph-mon[33333]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T11:52:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/100158745' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"} : dispatch 2024-09-21T11:52:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"} : dispatch 2024-09-21T11:52:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"}]': finished 2024-09-21T11:52:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:06 smithi193 ceph-mon[33333]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T11:52:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:06 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:07.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[30293]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T11:52:07.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/100158745' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"} : dispatch 2024-09-21T11:52:07.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"} : dispatch 2024-09-21T11:52:07.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"}]': finished 2024-09-21T11:52:07.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[30293]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T11:52:07.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:07.078 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[41097]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-21T11:52:07.078 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/100158745' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"} : dispatch 2024-09-21T11:52:07.078 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"} : dispatch 2024-09-21T11:52:07.078 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2576976-ac0c-45e0-8353-95220342328d"}]': finished 2024-09-21T11:52:07.078 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[41097]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T11:52:07.078 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:06 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:08.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/3687556744' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:52:08.035 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/3687556744' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:52:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/3687556744' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:52:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:08 smithi193 ceph-mon[33333]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T11:52:09.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:08 smithi107 ceph-mon[30293]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T11:52:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:08 smithi107 ceph-mon[41097]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T11:52:11.032 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:10 smithi193 ceph-mon[33333]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T11:52:11.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:10 smithi107 ceph-mon[30293]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T11:52:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:10 smithi107 ceph-mon[41097]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T11:52:12.850 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:12 smithi193 ceph-mon[33333]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:13.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:12 smithi107 ceph-mon[30293]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:12 smithi107 ceph-mon[41097]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:14 smithi193 ceph-mon[33333]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:14 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T11:52:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:14 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:14 smithi193 ceph-mon[33333]: Deploying daemon osd.5 on smithi193 2024-09-21T11:52:15.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[30293]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[30293]: Deploying daemon osd.5 on smithi193 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[41097]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:14 smithi107 ceph-mon[41097]: Deploying daemon osd.5 on smithi193 2024-09-21T11:52:16.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:16 smithi193 ceph-mon[33333]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:17.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:16 smithi107 ceph-mon[30293]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:16 smithi107 ceph-mon[41097]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:18 smithi193 ceph-mon[33333]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:19.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:18 smithi107 ceph-mon[30293]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:19.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:18 smithi107 ceph-mon[41097]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:20.889 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:20 smithi193 ceph-mon[33333]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:21.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:20 smithi107 ceph-mon[30293]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:20 smithi107 ceph-mon[41097]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:21 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:21 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:21 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:22.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:21 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:22.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:21 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:21 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:21 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:21 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:21 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:22 smithi193 ceph-mon[33333]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:23.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:22 smithi107 ceph-mon[30293]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:22 smithi107 ceph-mon[41097]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:25.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[30293]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[41097]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:25 smithi193 ceph-mon[33333]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:26.361 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-09-21T11:52:26.362 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.359+0000 7fec74ff9640 1 -- 172.21.15.193:0/2230895099 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec78063590 con 0x7fec6407b060 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fec6407b060 msgr2=0x7fec6407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fec6407b060 0x7fec6407d520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec600058c0 tx=0x7fec6000a4c0 comp rx=0 tx=0).stop 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 msgr2=0x7fec78198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78198360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fec68005940 tx=0x7fec68004360 comp rx=0 tx=0).stop 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 shutdown_connections 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fec6407b060 0x7fec6407d520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec78103400 0x7fec781995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec78102a40 0x7fec781989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 --2- 172.21.15.193:0/2230895099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec781018e0 0x7fec78198360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:26.364 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 >> 172.21.15.193:0/2230895099 conn(0x7fec780fd1b0 msgr2=0x7fec780feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:52:26.365 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 shutdown_connections 2024-09-21T11:52:26.365 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:26.362+0000 7fec7d593640 1 -- 172.21.15.193:0/2230895099 wait complete. 2024-09-21T11:52:27.021 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.5.service 2024-09-21T11:52:27.023 INFO:tasks.cephadm:Deploying osd.6 on smithi193 with /dev/vg_nvme/lv_2... 2024-09-21T11:52:27.023 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T11:52:27.265 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:27 smithi193 ceph-mon[33333]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:27.265 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:27 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:27.265 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:27 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:27.265 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:27 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:27.265 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:27 smithi193 ceph-mon[33333]: from='osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:52:27.265 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:27 smithi193 ceph-mon[33333]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[30293]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[30293]: from='osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[30293]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[41097]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:27.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[41097]: from='osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:52:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:27 smithi107 ceph-mon[41097]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:52:27.669 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-mon[33333]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T11:52:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-mon[33333]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-mon[33333]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T11:52:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-mon[33333]: from='osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-mon[33333]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:28.551 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T11:52:28.366+0000 7f5910c82640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T11:52:28.551 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 11:52:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T11:52:28.372+0000 7f590caa9640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:52:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[30293]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T11:52:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[30293]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[30293]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[30293]: from='osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[30293]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[41097]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[41097]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[41097]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[41097]: from='osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:28 smithi107 ceph-mon[41097]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:29.681 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:29 smithi193 ceph-mon[33333]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:52:29.681 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:29 smithi193 ceph-mon[33333]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T11:52:29.681 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:29.681 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:29 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:29.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[30293]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[30293]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[41097]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[41097]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:29 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:30.383 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] boot 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi193 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:52:30.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] boot 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi193 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[30293]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T11:52:30.609 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:52:30.609 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:52:30.609 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:52:30.609 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: osd.5 [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] boot 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi193 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.610 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:30 smithi193 ceph-mon[33333]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T11:52:30.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:30 smithi107 ceph-mon[41097]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T11:52:31.799 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:52:31.824 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-09-21T11:52:32.070 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:32.457 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:32 smithi193 ceph-mon[33333]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:32.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:32 smithi193 ceph-mon[33333]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T11:52:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 -- 172.21.15.193:0/383385263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c104990 msgr2=0x7fa05c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 --2- 172.21.15.193:0/383385263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c104990 0x7fa05c100ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa050009900 tx=0x7fa05002a410 comp rx=0 tx=0).stop 2024-09-21T11:52:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 -- 172.21.15.193:0/383385263 shutdown_connections 2024-09-21T11:52:32.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 --2- 172.21.15.193:0/383385263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa05c1011f0 0x7fa05c10b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 --2- 172.21.15.193:0/383385263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c104990 0x7fa05c100ad0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 --2- 172.21.15.193:0/383385263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c104050 0x7fa05c104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 -- 172.21.15.193:0/383385263 >> 172.21.15.193:0/383385263 conn(0x7fa05c0fc960 msgr2=0x7fa05c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 -- 172.21.15.193:0/383385263 shutdown_connections 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.539+0000 7fa0618ac640 1 -- 172.21.15.193:0/383385263 wait complete. 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 Processor -- start 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 -- start start 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 0x7fa05c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa05c104050 0x7fa05c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c104990 0x7fa05c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10db10 con 0x7fa05c104990 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c10d990 con 0x7fa05c1011f0 2024-09-21T11:52:32.544 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.540+0000 7fa0618ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa05c10dc90 con 0x7fa05c104050 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 0x7fa05c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 0x7fa05c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:52784/0 (socket says 172.21.15.193:52784) 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 -- 172.21.15.193:0/1375056587 learned_addr learned my addr 172.21.15.193:0/1375056587 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05b7fe640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c104990 0x7fa05c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05a7fc640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa05c104050 0x7fa05c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 -- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa05c104050 msgr2=0x7fa05c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa05c104050 0x7fa05c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 -- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c104990 msgr2=0x7fa05c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c104990 0x7fa05c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 -- 172.21.15.193:0/1375056587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c199e80 con 0x7fa05c1011f0 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa05affd640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 0x7fa05c198350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00eab0 tx=0x7fa04c00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00ce40 con 0x7fa05c1011f0 2024-09-21T11:52:32.545 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c19a170 con 0x7fa05c1011f0 2024-09-21T11:52:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa04c0118c0 con 0x7fa05c1011f0 2024-09-21T11:52:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010840 con 0x7fa05c1011f0 2024-09-21T11:52:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.541+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c1a0850 con 0x7fa05c1011f0 2024-09-21T11:52:32.546 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.542+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c104450 con 0x7fa05c1011f0 2024-09-21T11:52:32.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.543+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7fa04c011a70 con 0x7fa05c1011f0 2024-09-21T11:52:32.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.544+0000 7fa0608aa640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa04407b060 0x7fa04407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:52:32.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.544+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4423+0+0 (secure 0 0 0) 0x7fa04c09e0e0 con 0x7fa05c1011f0 2024-09-21T11:52:32.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.544+0000 7fa05a7fc640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa04407b060 0x7fa04407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:52:32.547 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.544+0000 7fa05a7fc640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa04407b060 0x7fa04407d520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa050005940 tx=0x7fa0500058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:52:32.549 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.547+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa04c016030 con 0x7fa05c1011f0 2024-09-21T11:52:32.737 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:32.735+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa05c063590 con 0x7fa04407b060 2024-09-21T11:52:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:32 smithi107 ceph-mon[41097]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:32 smithi107 ceph-mon[41097]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T11:52:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:32 smithi107 ceph-mon[30293]: pgmap v141: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:32 smithi107 ceph-mon[30293]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T11:52:33.719 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:33 smithi193 ceph-mon[33333]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:52:33.719 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:52:33.719 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:52:33.719 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:33 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[30293]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[41097]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:52:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:33 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:34.716 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:34 smithi193 ceph-mon[33333]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:34.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:34 smithi107 ceph-mon[30293]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:34 smithi107 ceph-mon[41097]: pgmap v143: 1 pgs: 1 remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/1021073283' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"} : dispatch 2024-09-21T11:52:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"} : dispatch 2024-09-21T11:52:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"}]': finished 2024-09-21T11:52:35.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:35 smithi193 ceph-mon[33333]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T11:52:35.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:35 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:35.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/1021073283' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"} : dispatch 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"} : dispatch 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"}]': finished 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[30293]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/1021073283' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"} : dispatch 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"} : dispatch 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14c736d5-64dc-41f2-9cc8-6b8adaba0ebc"}]': finished 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[41097]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T11:52:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:35 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:36.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:36 smithi107 ceph-mon[41097]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:36.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/2504791966' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:52:36.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:36 smithi107 ceph-mon[30293]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:36.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/2504791966' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:52:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:36 smithi193 ceph-mon[33333]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/2504791966' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:52:38.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:38 smithi193 ceph-mon[33333]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T11:52:38.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:38 smithi107 ceph-mon[30293]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T11:52:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:38 smithi107 ceph-mon[41097]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T11:52:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:40 smithi193 ceph-mon[33333]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:52:40.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:40 smithi107 ceph-mon[30293]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:52:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:40 smithi107 ceph-mon[41097]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:52:42.737 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:42 smithi193 ceph-mon[33333]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:42.737 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:42 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T11:52:42.737 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:42 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:42.737 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:42 smithi193 ceph-mon[33333]: Deploying daemon osd.6 on smithi193 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[30293]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[30293]: Deploying daemon osd.6 on smithi193 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[41097]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:42 smithi107 ceph-mon[41097]: Deploying daemon osd.6 on smithi193 2024-09-21T11:52:44.663 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:44 smithi193 ceph-mon[33333]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:44 smithi107 ceph-mon[30293]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:44 smithi107 ceph-mon[41097]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:52:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:46 smithi107 ceph-mon[30293]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:46 smithi107 ceph-mon[41097]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:46 smithi193 ceph-mon[33333]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:48.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:48 smithi107 ceph-mon[30293]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:48 smithi107 ceph-mon[41097]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:48 smithi193 ceph-mon[33333]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:50.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:50 smithi193 ceph-mon[33333]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:50.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:50 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:50.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:50 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:50.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:50 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[30293]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[41097]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:50 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:52 smithi107 ceph-mon[30293]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:52 smithi107 ceph-mon[41097]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:52 smithi193 ceph-mon[33333]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:54.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:53 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:53 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:53 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:53 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:53 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:53 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:55.201 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.198+0000 7fa0608aa640 1 -- 172.21.15.193:0/1375056587 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa05c063590 con 0x7fa04407b060 2024-09-21T11:52:55.201 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 6 on host 'smithi193' 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa04407b060 msgr2=0x7fa04407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa04407b060 0x7fa04407d520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa050005940 tx=0x7fa0500058d0 comp rx=0 tx=0).stop 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 msgr2=0x7fa05c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 0x7fa05c198350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00eab0 tx=0x7fa04c00ef70 comp rx=0 tx=0).stop 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 shutdown_connections 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa04407b060 0x7fa04407d520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa05c104990 0x7fa05c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa05c104050 0x7fa05c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 --2- 172.21.15.193:0/1375056587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa05c1011f0 0x7fa05c198350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 >> 172.21.15.193:0/1375056587 conn(0x7fa05c0fc960 msgr2=0x7fa05c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 shutdown_connections 2024-09-21T11:52:55.204 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:52:55.201+0000 7fa0618ac640 1 -- 172.21.15.193:0/1375056587 wait complete. 2024-09-21T11:52:55.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:55 smithi107 ceph-mon[30293]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:55 smithi107 ceph-mon[30293]: from='osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:52:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:55 smithi107 ceph-mon[30293]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:52:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:55 smithi107 ceph-mon[41097]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:55 smithi107 ceph-mon[41097]: from='osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:52:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:55 smithi107 ceph-mon[41097]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:52:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:55 smithi193 ceph-mon[33333]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:55 smithi193 ceph-mon[33333]: from='osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:52:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:55 smithi193 ceph-mon[33333]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:52:55.870 DEBUG:teuthology.orchestra.run.smithi193:osd.6> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.6.service 2024-09-21T11:52:55.872 INFO:tasks.cephadm:Deploying osd.7 on smithi193 with /dev/vg_nvme/lv_1... 2024-09-21T11:52:55.872 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[30293]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:52:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:56.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:56 smithi107 ceph-mon[41097]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:52:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T11:52:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T11:52:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:52:56.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:56.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:56.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:56.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-mon[33333]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:52:56.612 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:56.983 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T11:52:56.626+0000 7fb5b5c09640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T11:52:56.983 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 11:52:56 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T11:52:56.634+0000 7fb5ad214640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:52:57.280 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:57 smithi193 ceph-mon[33333]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:57.280 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:57 smithi193 ceph-mon[33333]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T11:52:57.280 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:57 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:57.280 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:57 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:57.280 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:57 smithi193 ceph-mon[33333]: from='osd.6 ' entity='osd.6' 2024-09-21T11:52:57.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[30293]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[30293]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[30293]: from='osd.6 ' entity='osd.6' 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[41097]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[41097]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:57 smithi107 ceph-mon[41097]: from='osd.6 ' entity='osd.6' 2024-09-21T11:52:58.469 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:58 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:52:58.469 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:58 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:52:58.469 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:58 smithi193 ceph-mon[33333]: osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] boot 2024-09-21T11:52:58.469 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:58 smithi193 ceph-mon[33333]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T11:52:58.470 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:58 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:58.470 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:58 smithi193 ceph-mon[33333]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T11:52:58.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[30293]: osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] boot 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[30293]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[30293]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[41097]: osd.6 [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] boot 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[41097]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:52:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:58 smithi107 ceph-mon[41097]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: osdmap e45: 7 total, 7 up, 7 in 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:59.499 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:59.500 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:52:59 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: osdmap e45: 7 total, 7 up, 7 in 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:52:59.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: osdmap e45: 7 total, 7 up, 7 in 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:52:59.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:52:59 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:00.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:00 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi193 2024-09-21T11:53:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:00 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T11:53:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:00 smithi107 ceph-mon[30293]: osdmap e46: 7 total, 7 up, 7 in 2024-09-21T11:53:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:00 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi193 2024-09-21T11:53:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:00 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T11:53:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:00 smithi107 ceph-mon[41097]: osdmap e46: 7 total, 7 up, 7 in 2024-09-21T11:53:00.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:00 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi193 2024-09-21T11:53:00.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:00 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T11:53:00.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:00 smithi193 ceph-mon[33333]: osdmap e46: 7 total, 7 up, 7 in 2024-09-21T11:53:00.789 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T11:53:00.815 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-09-21T11:53:01.059 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:01.439 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:01 smithi193 ceph-mon[33333]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:01 smithi107 ceph-mon[30293]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:01 smithi107 ceph-mon[41097]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:01.524 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.520+0000 7f04a1546640 1 -- 172.21.15.193:0/3396490185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c102a00 msgr2=0x7f049c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.520+0000 7f04a1546640 1 --2- 172.21.15.193:0/3396490185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c102a00 0x7f049c102e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f048c009930 tx=0x7f048c02a420 comp rx=0 tx=0).stop 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 -- 172.21.15.193:0/3396490185 shutdown_connections 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 --2- 172.21.15.193:0/3396490185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f049c104520 0x7f049c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 --2- 172.21.15.193:0/3396490185 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f049c103b60 0x7f049c103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 --2- 172.21.15.193:0/3396490185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c102a00 0x7f049c102e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 -- 172.21.15.193:0/3396490185 >> 172.21.15.193:0/3396490185 conn(0x7f049c0fe2f0 msgr2=0x7f049c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 -- 172.21.15.193:0/3396490185 shutdown_connections 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 -- 172.21.15.193:0/3396490185 wait complete. 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 Processor -- start 2024-09-21T11:53:01.525 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 -- start start 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f049c102a00 0x7f049c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 0x7f049c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.521+0000 7f04a1546640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f049c104520 0x7f049c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f04a1546640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10dc20 con 0x7f049c102a00 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f04a1546640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10daa0 con 0x7f049c103b60 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f04a1546640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f049c10dda0 con 0x7f049c104520 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 0x7f049c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 0x7f049c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53348/0 (socket says 172.21.15.193:53348) 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 -- 172.21.15.193:0/504901709 learned_addr learned my addr 172.21.15.193:0/504901709 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049affd640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f049c102a00 0x7f049c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049b7fe640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f049c104520 0x7f049c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 -- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f049c104520 msgr2=0x7f049c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:01.526 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f049c104520 0x7f049c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:01.527 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 -- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f049c102a00 msgr2=0x7f049c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:01.527 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f049c102a00 0x7f049c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:01.527 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 -- 172.21.15.193:0/504901709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c199fd0 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f049a7fc640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 0x7f049c198af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f049000b4a0 tx=0x7f049000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490014070 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0490010a60 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c19a2c0 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900179e0 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.522+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c1a09a0 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.523+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049c102e80 con 0x7f049c103b60 2024-09-21T11:53:01.528 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.524+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f0490017070 con 0x7f049c103b60 2024-09-21T11:53:01.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.525+0000 7f0483fff640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f048407b060 0x7f048407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:01.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.525+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4855+0+0 (secure 0 0 0) 0x7f049009cfd0 con 0x7f049c103b60 2024-09-21T11:53:01.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.525+0000 7f049affd640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f048407b060 0x7f048407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:01.529 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.525+0000 7f049affd640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f048407b060 0x7f048407d520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f048c005ab0 tx=0x7f048c005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:01.532 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.529+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0490068ae0 con 0x7f049c103b60 2024-09-21T11:53:01.721 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:01.718+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f049c02e7c0 con 0x7f048407b060 2024-09-21T11:53:02.348 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:53:02.349 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:53:02.349 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:53:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:53:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:53:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:53:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:03.357 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:03 smithi193 ceph-mon[33333]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:03.357 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:03 smithi193 ceph-mon[33333]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:03.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:03 smithi107 ceph-mon[30293]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:03.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:03 smithi107 ceph-mon[30293]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:03.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:03 smithi107 ceph-mon[41097]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:03.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:03 smithi107 ceph-mon[41097]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/2548881137' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"} : dispatch 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"} : dispatch 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"}]': finished 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[30293]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/2548881137' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"} : dispatch 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"} : dispatch 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"}]': finished 2024-09-21T11:53:04.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[41097]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T11:53:04.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:04 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/2548881137' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"} : dispatch 2024-09-21T11:53:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"} : dispatch 2024-09-21T11:53:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb16cb03-fe65-4189-818c-29a99c0db388"}]': finished 2024-09-21T11:53:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:04 smithi193 ceph-mon[33333]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T11:53:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:04 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:05.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:05 smithi107 ceph-mon[41097]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:05.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/2553052511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:53:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:05 smithi107 ceph-mon[30293]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/2553052511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:53:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:05 smithi193 ceph-mon[33333]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/2553052511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:53:07.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:07 smithi107 ceph-mon[30293]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:07 smithi107 ceph-mon[41097]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:07 smithi193 ceph-mon[33333]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:09.279 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:09 smithi193 ceph-mon[33333]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:09.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:09 smithi107 ceph-mon[30293]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:09 smithi107 ceph-mon[41097]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:11.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:11 smithi107 ceph-mon[30293]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T11:53:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:11 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:11 smithi107 ceph-mon[41097]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:11 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T11:53:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:11 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:11.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:11 smithi193 ceph-mon[33333]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:11.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:11 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T11:53:11.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:11 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:12.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:12 smithi107 ceph-mon[30293]: Deploying daemon osd.7 on smithi193 2024-09-21T11:53:12.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:12 smithi107 ceph-mon[41097]: Deploying daemon osd.7 on smithi193 2024-09-21T11:53:12.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:12 smithi193 ceph-mon[33333]: Deploying daemon osd.7 on smithi193 2024-09-21T11:53:13.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:13 smithi193 ceph-mon[33333]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:13.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:13 smithi107 ceph-mon[30293]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:13.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:13 smithi107 ceph-mon[41097]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:15.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:15 smithi193 ceph-mon[33333]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:15.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:15 smithi107 ceph-mon[30293]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:15 smithi107 ceph-mon[41097]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:17 smithi193 ceph-mon[33333]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:17.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:17 smithi107 ceph-mon[30293]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:17.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:17 smithi107 ceph-mon[41097]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:19.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:19 smithi193 ceph-mon[33333]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:19.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:19 smithi107 ceph-mon[30293]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:19 smithi107 ceph-mon[41097]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:20.528 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:20.528 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:20.528 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:20 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:20.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:20.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:20 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:20 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:20 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:20 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:21.668 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:21 smithi193 ceph-mon[33333]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:21 smithi107 ceph-mon[30293]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:21.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:21 smithi107 ceph-mon[41097]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[30293]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[41097]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:23.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:23.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:23 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:23 smithi193 ceph-mon[33333]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:23 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:23 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:23.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:23 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:23.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:23 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:23.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:23 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:24.426 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.424+0000 7f0483fff640 1 -- 172.21.15.193:0/504901709 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f049c02e7c0 con 0x7f048407b060 2024-09-21T11:53:24.426 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 7 on host 'smithi193' 2024-09-21T11:53:24.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.426+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f048407b060 msgr2=0x7f048407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:24.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.426+0000 7f04a1546640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f048407b060 0x7f048407d520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f048c005ab0 tx=0x7f048c005a40 comp rx=0 tx=0).stop 2024-09-21T11:53:24.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.426+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 msgr2=0x7f049c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:24.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.426+0000 7f04a1546640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 0x7f049c198af0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f049000b4a0 tx=0x7f049000b960 comp rx=0 tx=0).stop 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 shutdown_connections 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f048407b060 0x7f048407d520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f049c104520 0x7f049c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f049c103b60 0x7f049c198af0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 --2- 172.21.15.193:0/504901709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f049c102a00 0x7f049c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 >> 172.21.15.193:0/504901709 conn(0x7f049c0fe2f0 msgr2=0x7f049c100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 shutdown_connections 2024-09-21T11:53:24.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:24.427+0000 7f04a1546640 1 -- 172.21.15.193:0/504901709 wait complete. 2024-09-21T11:53:25.068 DEBUG:teuthology.orchestra.run.smithi193:osd.7> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.7.service 2024-09-21T11:53:25.071 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-21T11:53:25.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd stat -f json 2024-09-21T11:53:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:25 smithi193 ceph-mon[33333]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:25 smithi193 ceph-mon[33333]: from='osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:53:25.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:25 smithi193 ceph-mon[33333]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:53:25.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:25.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:25.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:25 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:25.318 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[30293]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[30293]: from='osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[30293]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[41097]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[41097]: from='osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:53:25.439 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[41097]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:53:25.440 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:25.440 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:25.440 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:25 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:25.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 -- 172.21.15.107:0/1978883384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458103430 msgr2=0x7f6458109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:25.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 --2- 172.21.15.107:0/1978883384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458103430 0x7f6458109ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6450009ca0 tx=0x7f645002a8c0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 -- 172.21.15.107:0/1978883384 shutdown_connections 2024-09-21T11:53:25.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 --2- 172.21.15.107:0/1978883384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458103430 0x7f6458109ee0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 --2- 172.21.15.107:0/1978883384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6458102820 0x7f6458102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 --2- 172.21.15.107:0/1978883384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6458101ee0 0x7f64581022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 -- 172.21.15.107:0/1978883384 >> 172.21.15.107:0/1978883384 conn(0x7f64580fc940 msgr2=0x7f64580fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 -- 172.21.15.107:0/1978883384 shutdown_connections 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 -- 172.21.15.107:0/1978883384 wait complete. 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.799+0000 7f645e465640 1 Processor -- start 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 -- start start 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 0x7f64581014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6458102820 0x7f645819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6458103430 0x7f645819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f645810c9b0 con 0x7f6458103430 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f645810c830 con 0x7f6458102820 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645e465640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f645810cb30 con 0x7f6458101ee0 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 0x7f64581014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f645ca02640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6458103430 0x7f645819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:25.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 0x7f64581014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:42240/0 (socket says 172.21.15.107:42240) 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 -- 172.21.15.107:0/1330662117 learned_addr learned my addr 172.21.15.107:0/1330662117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f644ffff640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6458102820 0x7f645819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 -- 172.21.15.107:0/1330662117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6458102820 msgr2=0x7f645819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6458102820 0x7f645819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 -- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6458103430 msgr2=0x7f645819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6458103430 0x7f645819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 -- 172.21.15.107:0/1330662117 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645819fe70 con 0x7f6458101ee0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.800+0000 7f6457fff640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 0x7f64581014b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f644800e9f0 tx=0x7f644800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.801+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644800cd80 con 0x7f6458101ee0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.801+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6448010880 con 0x7f6458101ee0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.801+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6458196df0 con 0x7f6458101ee0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.801+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448018760 con 0x7f6458101ee0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.801+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6458197380 con 0x7f6458101ee0 2024-09-21T11:53:25.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.802+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64581022e0 con 0x7f6458101ee0 2024-09-21T11:53:25.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.802+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f6448010430 con 0x7f6458101ee0 2024-09-21T11:53:25.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.803+0000 7f6455ffb640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f644407b060 0x7f644407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:25.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.803+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 4991+0+0 (secure 0 0 0) 0x7f644809e0c0 con 0x7f6458101ee0 2024-09-21T11:53:25.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.803+0000 7f644ffff640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f644407b060 0x7f644407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:25.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.803+0000 7f644ffff640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f644407b060 0x7f644407d520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6440009bd0 tx=0x7f6440005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:25.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.807+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6448069c90 con 0x7f6458101ee0 2024-09-21T11:53:25.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.985+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f645802e8d0 con 0x7f6458101ee0 2024-09-21T11:53:25.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.986+0000 7f6455ffb640 1 -- 172.21.15.107:0/1330662117 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v48) ==== 74+0+130 (secure 0 0 0) 0x7f644806ae20 con 0x7f6458101ee0 2024-09-21T11:53:25.988 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:53:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f644407b060 msgr2=0x7f644407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:25.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f644407b060 0x7f644407d520 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6440009bd0 tx=0x7f6440005f90 comp rx=0 tx=0).stop 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 msgr2=0x7f64581014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 0x7f64581014b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f644800e9f0 tx=0x7f644800eeb0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 shutdown_connections 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f644407b060 0x7f644407d520 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6458103430 0x7f645819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6458102820 0x7f645819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 --2- 172.21.15.107:0/1330662117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6458101ee0 0x7f64581014b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.988+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 >> 172.21.15.107:0/1330662117 conn(0x7f64580fc940 msgr2=0x7f64580fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.989+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 shutdown_connections 2024-09-21T11:53:25.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:25.989+0000 7f645e465640 1 -- 172.21.15.107:0/1330662117 wait complete. 2024-09-21T11:53:26.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[30293]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[30293]: osdmap e48: 8 total, 7 up, 8 in 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[30293]: from='osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[30293]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1330662117' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[41097]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[41097]: osdmap e48: 8 total, 7 up, 8 in 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[41097]: from='osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:53:26.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[41097]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:53:26.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1330662117' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:53:26.331 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-mon[33333]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T11:53:26.331 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-mon[33333]: osdmap e48: 8 total, 7 up, 8 in 2024-09-21T11:53:26.331 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:26.331 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-mon[33333]: from='osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:53:26.331 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-mon[33333]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T11:53:26.331 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1330662117' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:53:26.649 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":48,"num_osds":8,"num_up_osds":7,"osd_up_since":1726919577,"num_in_osds":8,"osd_in_since":1726919584,"num_remapped_pgs":0} 2024-09-21T11:53:26.884 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T11:53:26.650+0000 7f7a61648640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T11:53:26.884 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 11:53:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T11:53:26.656+0000 7f7a59467640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: osdmap e49: 8 total, 7 up, 8 in 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: from='osd.7 ' entity='osd.7' 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] boot 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T11:53:27.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: osdmap e49: 8 total, 7 up, 8 in 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: from='osd.7 ' entity='osd.7' 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] boot 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T11:53:27.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:27 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:53:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T11:53:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: osdmap e49: 8 total, 7 up, 8 in 2024-09-21T11:53:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: from='osd.7 ' entity='osd.7' 2024-09-21T11:53:27.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: osd.7 [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] boot 2024-09-21T11:53:27.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T11:53:27.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:27 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:53:27.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd stat -f json 2024-09-21T11:53:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:28.333 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: purged_snaps scrub starts 2024-09-21T11:53:28.333 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: purged_snaps scrub ok 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[30293]: osdmap e51: 8 total, 8 up, 8 in 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: purged_snaps scrub starts 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: purged_snaps scrub ok 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:28.335 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:28.336 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.336 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:28 smithi107 ceph-mon[41097]: osdmap e51: 8 total, 8 up, 8 in 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 -- 172.21.15.107:0/134410170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038102a70 msgr2=0x7f2038102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 --2- 172.21.15.107:0/134410170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038102a70 0x7f2038102ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f20280098c0 tx=0x7f202802a390 comp rx=0 tx=0).stop 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 -- 172.21.15.107:0/134410170 shutdown_connections 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 --2- 172.21.15.107:0/134410170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2038103430 0x7f2038109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 --2- 172.21.15.107:0/134410170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038102a70 0x7f2038102ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 --2- 172.21.15.107:0/134410170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038101910 0x7f2038101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 -- 172.21.15.107:0/134410170 >> 172.21.15.107:0/134410170 conn(0x7f20380fd1e0 msgr2=0x7f20380ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 -- 172.21.15.107:0/134410170 shutdown_connections 2024-09-21T11:53:28.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 -- 172.21.15.107:0/134410170 wait complete. 2024-09-21T11:53:28.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.336+0000 7f203f8a7640 1 Processor -- start 2024-09-21T11:53:28.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 -- start start 2024-09-21T11:53:28.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 0x7f20381972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:28.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038102a70 0x7f2038197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2038103430 0x7f2038198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f203810ca70 con 0x7f2038103430 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f203810c8f0 con 0x7f2038102a70 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203f8a7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f203810cbf0 con 0x7f2038101910 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 0x7f20381972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 0x7f20381972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:42258/0 (socket says 172.21.15.107:42258) 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 -- 172.21.15.107:0/2166148064 learned_addr learned my addr 172.21.15.107:0/2166148064 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203de44640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2038103430 0x7f2038198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203ce42640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038102a70 0x7f2038197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 -- 172.21.15.107:0/2166148064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038102a70 msgr2=0x7f2038197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038102a70 0x7f2038197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 -- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2038103430 msgr2=0x7f2038198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2038103430 0x7f2038198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 -- 172.21.15.107:0/2166148064 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038198e20 con 0x7f2038101910 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f203d643640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 0x7f20381972f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f203400b4a0 tx=0x7f203400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.337+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034014070 con 0x7f2038101910 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.338+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2034010a60 con 0x7f2038101910 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.338+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2038199110 con 0x7f2038101910 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.338+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20340179e0 con 0x7f2038101910 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.338+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f203806b4f0 con 0x7f2038101910 2024-09-21T11:53:28.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.338+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2038101d10 con 0x7f2038101910 2024-09-21T11:53:28.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.339+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f2034017070 con 0x7f2038101910 2024-09-21T11:53:28.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.340+0000 7f202e7fc640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f202407b060 0x7f202407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:28.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.340+0000 7f203ce42640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f202407b060 0x7f202407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:28.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.340+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 5287+0+0 (secure 0 0 0) 0x7f203409d190 con 0x7f2038101910 2024-09-21T11:53:28.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.340+0000 7f203ce42640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f202407b060 0x7f202407d520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2028035040 tx=0x7f2028000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:28.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.344+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2034068af0 con 0x7f2038101910 2024-09-21T11:53:28.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.521+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f203806c100 con 0x7f2038101910 2024-09-21T11:53:28.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.522+0000 7f202e7fc640 1 -- 172.21.15.107:0/2166148064 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v51) ==== 74+0+130 (secure 0 0 0) 0x7f203406d9a0 con 0x7f2038101910 2024-09-21T11:53:28.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f202407b060 msgr2=0x7f202407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f202407b060 0x7f202407d520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2028035040 tx=0x7f2028000bc0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 msgr2=0x7f20381972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 0x7f20381972f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f203400b4a0 tx=0x7f203400b960 comp rx=0 tx=0).stop 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 shutdown_connections 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f202407b060 0x7f202407d520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2038103430 0x7f2038198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038102a70 0x7f2038197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 --2- 172.21.15.107:0/2166148064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2038101910 0x7f20381972f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 >> 172.21.15.107:0/2166148064 conn(0x7f20380fd1e0 msgr2=0x7f20380feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 shutdown_connections 2024-09-21T11:53:28.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:28.524+0000 7f203f8a7640 1 -- 172.21.15.107:0/2166148064 wait complete. 2024-09-21T11:53:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: purged_snaps scrub starts 2024-09-21T11:53:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: purged_snaps scrub ok 2024-09-21T11:53:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:28.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:28 smithi193 ceph-mon[33333]: osdmap e51: 8 total, 8 up, 8 in 2024-09-21T11:53:29.306 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1726919607,"num_in_osds":8,"osd_in_since":1726919584,"num_remapped_pgs":0} 2024-09-21T11:53:29.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T11:53:29.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[30293]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[30293]: Detected new or changed devices on smithi193 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2166148064' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[30293]: osdmap e52: 8 total, 8 up, 8 in 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[41097]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[41097]: Detected new or changed devices on smithi193 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2166148064' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:53:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:29 smithi107 ceph-mon[41097]: osdmap e52: 8 total, 8 up, 8 in 2024-09-21T11:53:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:29 smithi193 ceph-mon[33333]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:29 smithi193 ceph-mon[33333]: Detected new or changed devices on smithi193 2024-09-21T11:53:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:29 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T11:53:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2166148064' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:53:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:29 smithi193 ceph-mon[33333]: osdmap e52: 8 total, 8 up, 8 in 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:30.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 -- 172.21.15.107:0/3580568692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 msgr2=0x7f916810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 --2- 172.21.15.107:0/3580568692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 0x7f916810e7f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9160009be0 tx=0x7f916002a6c0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 -- 172.21.15.107:0/3580568692 shutdown_connections 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 --2- 172.21.15.107:0/3580568692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 0x7f916810e7f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 --2- 172.21.15.107:0/3580568692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168107670 0x7f9168107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 --2- 172.21.15.107:0/3580568692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168101c60 0x7f9168102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 -- 172.21.15.107:0/3580568692 >> 172.21.15.107:0/3580568692 conn(0x7f91680fd6d0 msgr2=0x7f91680ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 -- 172.21.15.107:0/3580568692 shutdown_connections 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 -- 172.21.15.107:0/3580568692 wait complete. 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 Processor -- start 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.019+0000 7f916f756640 1 -- start start 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 0x7f91681995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:30.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168107670 0x7f9168199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 0x7f916819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111180 con 0x7f9168101c60 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111000 con 0x7f9168107670 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9168111300 con 0x7f9168107ff0 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 0x7f91681995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 0x7f91681995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39758/0 (socket says 172.21.15.107:39758) 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 -- 172.21.15.107:0/1345348607 learned_addr learned my addr 172.21.15.107:0/1345348607 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916dcf3640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 0x7f916819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916ccf1640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168107670 0x7f9168199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 -- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 msgr2=0x7f916819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 0x7f916819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 -- 172.21.15.107:0/1345348607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168107670 msgr2=0x7f9168199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168107670 0x7f9168199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 -- 172.21.15.107:0/1345348607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91681a49a0 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916ccf1640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168107670 0x7f9168199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916d4f2640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 0x7f91681995a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f915800b4a0 tx=0x7f915800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158014070 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9158010a60 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91681a4c90 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91580179e0 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.020+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91681a5240 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.021+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9168102040 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.022+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f9158017070 con 0x7f9168101c60 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.022+0000 7f915e7fc640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f915407b060 0x7f915407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:30.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.022+0000 7f916ccf1640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f915407b060 0x7f915407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:30.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.022+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5287+0+0 (secure 0 0 0) 0x7f915809d090 con 0x7f9168101c60 2024-09-21T11:53:30.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.023+0000 7f916ccf1640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f915407b060 0x7f915407d520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f916806b6a0 tx=0x7f9164009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:30.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.027+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9158068a70 con 0x7f9168101c60 2024-09-21T11:53:30.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.206+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9168109840 con 0x7f9168101c60 2024-09-21T11:53:30.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.207+0000 7f915e7fc640 1 -- 172.21.15.107:0/1345348607 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13625 (secure 0 0 0) 0x7f915806d920 con 0x7f9168101c60 2024-09-21T11:53:30.209 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:53:30.209 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":52,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","created":"2024-09-21T11:47:57.300122+0000","modified":"2024-09-21T11:53:29.210319+0000","last_up_change":"2024-09-21T11:53:27.207544+0000","last_in_change":"2024-09-21T11:53:04.078828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:51:05.622296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e4feb96-c3c7-4f2d-9ba3-1db963213296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6803","nonce":2032034052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6805","nonce":2032034052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6809","nonce":2032034052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6807","nonce":2032034052}]},"public_addr":"172.21.15.107:6803/2032034052","cluster_addr":"172.21.15.107:6805/2032034052","heartbeat_back_addr":"172.21.15.107:6809/2032034052","heartbeat_front_addr":"172.21.15.107:6807/2032034052","state":["exists","up"]},{"osd":1,"uuid":"a464ccec-f1a1-48ae-b233-89519142c774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6811","nonce":3491646662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6813","nonce":3491646662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6817","nonce":3491646662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6815","nonce":3491646662}]},"public_addr":"172.21.15.107:6811/3491646662","cluster_addr":"172.21.15.107:6813/3491646662","heartbeat_back_addr":"172.21.15.107:6817/3491646662","heartbeat_front_addr":"172.21.15.107:6815/3491646662","state":["exists","up"]},{"osd":2,"uuid":"74584ac0-8833-4c1e-9f98-2bef0780286e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6819","nonce":216243970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6821","nonce":216243970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6825","nonce":216243970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6823","nonce":216243970}]},"public_addr":"172.21.15.107:6819/216243970","cluster_addr":"172.21.15.107:6821/216243970","heartbeat_back_addr":"172.21.15.107:6825/216243970","heartbeat_front_addr":"172.21.15.107:6823/216243970","state":["exists","up"]},{"osd":3,"uuid":"661db9ba-cabf-4159-b051-379aa09acc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6827","nonce":1625163749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6829","nonce":1625163749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6833","nonce":1625163749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6831","nonce":1625163749}]},"public_addr":"172.21.15.107:6827/1625163749","cluster_addr":"172.21.15.107:6829/1625163749","heartbeat_back_addr":"172.21.15.107:6833/1625163749","heartbeat_front_addr":"172.21.15.107:6831/1625163749","state":["exists","up"]},{"osd":4,"uuid":"6c71418f-c007-4428-b338-3ebfc82f779d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6801","nonce":3995503105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6803","nonce":3995503105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6807","nonce":3995503105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6805","nonce":3995503105}]},"public_addr":"172.21.15.193:6801/3995503105","cluster_addr":"172.21.15.193:6803/3995503105","heartbeat_back_addr":"172.21.15.193:6807/3995503105","heartbeat_front_addr":"172.21.15.193:6805/3995503105","state":["exists","up"]},{"osd":5,"uuid":"f2576976-ac0c-45e0-8353-95220342328d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6809","nonce":102264086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6811","nonce":102264086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6815","nonce":102264086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6813","nonce":102264086}]},"public_addr":"172.21.15.193:6809/102264086","cluster_addr":"172.21.15.193:6811/102264086","heartbeat_back_addr":"172.21.15.193:6815/102264086","heartbeat_front_addr":"172.21.15.193:6813/102264086","state":["exists","up"]},{"osd":6,"uuid":"14c736d5-64dc-41f2-9cc8-6b8adaba0ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6817","nonce":1896699774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6819","nonce":1896699774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6823","nonce":1896699774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6821","nonce":1896699774}]},"public_addr":"172.21.15.193:6817/1896699774","cluster_addr":"172.21.15.193:6819/1896699774","heartbeat_back_addr":"172.21.15.193:6823/1896699774","heartbeat_front_addr":"172.21.15.193:6821/1896699774","state":["exists","up"]},{"osd":7,"uuid":"fb16cb03-fe65-4189-818c-29a99c0db388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6825","nonce":338565813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6827","nonce":338565813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6831","nonce":338565813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6829","nonce":338565813}]},"public_addr":"172.21.15.193:6825/338565813","cluster_addr":"172.21.15.193:6827/338565813","heartbeat_back_addr":"172.21.15.193:6831/338565813","heartbeat_front_addr":"172.21.15.193:6829/338565813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:50:02.844467+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:50:32.631011+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:02.098307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:31.483117+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:58.119895+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:52:27.691211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:52:56.093529+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:53:25.370332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3723772931":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6800/1807990149":"2024-09-22T11:48:45.320004+0000","172.21.15.107:0/1627664470":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6800/1943709907":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/3703659764":"2024-09-22T11:48:45.320004+0000","172.21.15.107:0/545710521":"2024-09-22T11:48:19.247117+0000","172.21.15.107:6801/1807990149":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6801/1943709907":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/2867069233":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/1351066913":"2024-09-22T11:48:19.247117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T11:53:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f915407b060 msgr2=0x7f915407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f915407b060 0x7f915407d520 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f916806b6a0 tx=0x7f9164009260 comp rx=0 tx=0).stop 2024-09-21T11:53:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 msgr2=0x7f91681995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 0x7f91681995a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f915800b4a0 tx=0x7f915800b960 comp rx=0 tx=0).stop 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 shutdown_connections 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f915407b060 0x7f915407d520 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9168107ff0 0x7f916819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9168107670 0x7f9168199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 --2- 172.21.15.107:0/1345348607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9168101c60 0x7f91681995a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.209+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 >> 172.21.15.107:0/1345348607 conn(0x7f91680fd6d0 msgr2=0x7f91680ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.210+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 shutdown_connections 2024-09-21T11:53:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:30.210+0000 7f916f756640 1 -- 172.21.15.107:0/1345348607 wait complete. 2024-09-21T11:53:30.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1345348607' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:53:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1345348607' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:53:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1345348607' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:53:30.838 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:51:05.622296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T11:53:30.839 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T11:53:31.080 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:31.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:31 smithi107 ceph-mon[30293]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:31.452 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:31 smithi107 ceph-mon[41097]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:31.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 -- 172.21.15.107:0/2960637416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c102d80 msgr2=0x7f0f1c103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:31.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 --2- 172.21.15.107:0/2960637416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c102d80 0x7f0f1c103160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009900 tx=0x7f0f1802a410 comp rx=0 tx=0).stop 2024-09-21T11:53:31.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 -- 172.21.15.107:0/2960637416 shutdown_connections 2024-09-21T11:53:31.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 --2- 172.21.15.107:0/2960637416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 0x7f0f1c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 --2- 172.21.15.107:0/2960637416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c108790 0x7f0f1c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 --2- 172.21.15.107:0/2960637416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c102d80 0x7f0f1c103160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 -- 172.21.15.107:0/2960637416 >> 172.21.15.107:0/2960637416 conn(0x7f0f1c0fe7d0 msgr2=0x7f0f1c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 -- 172.21.15.107:0/2960637416 shutdown_connections 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.529+0000 7f0f238a5640 1 -- 172.21.15.107:0/2960637416 wait complete. 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 Processor -- start 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 -- start start 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 0x7f0f1c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c108790 0x7f0f1c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 0x7f0f1c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:31.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c112260 con 0x7f0f1c108790 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c1120e0 con 0x7f0f1c109110 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f238a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1c1123e0 con 0x7f0f1c102d80 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 0x7f0f1c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 0x7f0f1c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:42304/0 (socket says 172.21.15.107:42304) 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 -- 172.21.15.107:0/3284931585 learned_addr learned my addr 172.21.15.107:0/3284931585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21e42640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 0x7f0f1c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f20e40640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c108790 0x7f0f1c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 -- 172.21.15.107:0/3284931585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 msgr2=0x7f0f1c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 0x7f0f1c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 -- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c108790 msgr2=0x7f0f1c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c108790 0x7f0f1c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 -- 172.21.15.107:0/3284931585 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c19e860 con 0x7f0f1c102d80 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f20e40640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c108790 0x7f0f1c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:53:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21e42640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 0x7f0f1c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.530+0000 7f0f21641640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 0x7f0f1c1987b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005940 tx=0x7f0f18032b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.531+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1803f030 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.531+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f0f18003a40 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.531+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c19eaf0 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.531+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18035070 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.531+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c1a5210 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.532+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c103160 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.532+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f0f18042030 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.533+0000 7f0f127fc640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0f0807b060 0x7f0f0807d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.533+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 5287+0+0 (secure 0 0 0) 0x7f0f180be320 con 0x7f0f1c102d80 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.533+0000 7f0f20e40640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0f0807b060 0x7f0f0807d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.533+0000 7f0f20e40640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0f0807b060 0x7f0f0807d520 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c102100 tx=0x7f0f0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.537+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f18085920 con 0x7f0f1c102d80 2024-09-21T11:53:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:31 smithi193 ceph-mon[33333]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:31.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.714+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f1c10aef0 con 0x7f0f1c102d80 2024-09-21T11:53:31.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.714+0000 7f0f127fc640 1 -- 172.21.15.107:0/3284931585 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f0f18089d00 con 0x7f0f1c102d80 2024-09-21T11:53:31.716 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.716+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0f0807b060 msgr2=0x7f0f0807d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.716+0000 7f0f238a5640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0f0807b060 0x7f0f0807d520 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c102100 tx=0x7f0f0c005f90 comp rx=0 tx=0).stop 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.716+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 msgr2=0x7f0f1c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.716+0000 7f0f238a5640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 0x7f0f1c1987b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005940 tx=0x7f0f18032b70 comp rx=0 tx=0).stop 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 shutdown_connections 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f0f0807b060 0x7f0f0807d520 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f1c109110 0x7f0f1c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f1c108790 0x7f0f1c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 --2- 172.21.15.107:0/3284931585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f1c102d80 0x7f0f1c1987b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 >> 172.21.15.107:0/3284931585 conn(0x7f0f1c0fe7d0 msgr2=0x7f0f1c100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 shutdown_connections 2024-09-21T11:53:31.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:31.717+0000 7f0f238a5640 1 -- 172.21.15.107:0/3284931585 wait complete. 2024-09-21T11:53:32.401 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3284931585' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T11:53:32.424 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3284931585' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T11:53:32.436 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi107 2024-09-21T11:53:32.436 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply rgw foo.a --placement '1;smithi107=foo.a' 2024-09-21T11:53:32.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3284931585' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T11:53:32.695 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 -- 172.21.15.193:0/1964942532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308102a10 msgr2=0x7fa308102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 --2- 172.21.15.193:0/1964942532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308102a10 0x7fa308102e10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009930 tx=0x7fa2fc02a420 comp rx=0 tx=0).stop 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 -- 172.21.15.193:0/1964942532 shutdown_connections 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 --2- 172.21.15.193:0/1964942532 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa308104530 0x7fa30810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 --2- 172.21.15.193:0/1964942532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa308103b70 0x7fa308103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 --2- 172.21.15.193:0/1964942532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308102a10 0x7fa308102e10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 -- 172.21.15.193:0/1964942532 >> 172.21.15.193:0/1964942532 conn(0x7fa3080fe320 msgr2=0x7fa308100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:33.147 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 -- 172.21.15.193:0/1964942532 shutdown_connections 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 -- 172.21.15.193:0/1964942532 wait complete. 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 Processor -- start 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.124+0000 7fa310135640 1 -- start start 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa308102a10 0x7fa308198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa308103b70 0x7fa308198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 0x7fa3081994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810dae0 con 0x7fa308103b70 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa30810d960 con 0x7fa308104530 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa30810dc60 con 0x7fa308102a10 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 0x7fa3081994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 0x7fa3081994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:57910/0 (socket says 172.21.15.193:57910) 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 -- 172.21.15.193:0/3023892944 learned_addr learned my addr 172.21.15.193:0/3023892944 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:33.148 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30ded1640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa308102a10 0x7fa308198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30d6d0640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa308103b70 0x7fa308198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 -- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa308102a10 msgr2=0x7fa308198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa308102a10 0x7fa308198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 -- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa308103b70 msgr2=0x7fa308198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa308103b70 0x7fa308198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 -- 172.21.15.193:0/3023892944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa308199d50 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa30e6d2640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 0x7fa3081994b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa30000cce0 tx=0x7fa30000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30000f450 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30819a040 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa30000fa40 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300018910 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.125+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081a0720 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.126+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa308102e10 con 0x7fa308104530 2024-09-21T11:53:33.149 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.127+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7fa30000f5f0 con 0x7fa308104530 2024-09-21T11:53:33.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.128+0000 7fa2faffd640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa2f407b060 0x7fa2f407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:33.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.128+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5287+0+0 (secure 0 0 0) 0x7fa30009e320 con 0x7fa308104530 2024-09-21T11:53:33.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.128+0000 7fa30ded1640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa2f407b060 0x7fa2f407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:33.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.128+0000 7fa30ded1640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa2f407b060 0x7fa2f407d520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc000c00 tx=0x7fa2fc009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:33.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.132+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa300014030 con 0x7fa308104530 2024-09-21T11:53:33.326 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.324+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}) -- 0x7fa30802e780 con 0x7fa2f407b060 2024-09-21T11:53:33.333 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.331+0000 7fa2faffd640 1 -- 172.21.15.193:0/3023892944 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fa30802e780 con 0x7fa2f407b060 2024-09-21T11:53:33.333 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled rgw.foo.a update... 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa2f407b060 msgr2=0x7fa2f407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa2f407b060 0x7fa2f407d520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc000c00 tx=0x7fa2fc009610 comp rx=0 tx=0).stop 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 msgr2=0x7fa3081994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 0x7fa3081994b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa30000cce0 tx=0x7fa30000a620 comp rx=0 tx=0).stop 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 shutdown_connections 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fa2f407b060 0x7fa2f407d520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa308104530 0x7fa3081994b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa308103b70 0x7fa308198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.336 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 --2- 172.21.15.193:0/3023892944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa308102a10 0x7fa308198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:33.337 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 >> 172.21.15.193:0/3023892944 conn(0x7fa3080fe320 msgr2=0x7fa3080ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:33.337 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 shutdown_connections 2024-09-21T11:53:33.337 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:33.334+0000 7fa310135640 1 -- 172.21.15.193:0/3023892944 wait complete. 2024-09-21T11:53:33.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:33 smithi107 ceph-mon[30293]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:33 smithi107 ceph-mon[41097]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:33.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:33 smithi193 ceph-mon[33333]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 380 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:33.945 DEBUG:teuthology.orchestra.run.smithi107:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@rgw.foo.a.service 2024-09-21T11:53:33.947 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi193 2024-09-21T11:53:33.948 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd pool create datapool 3 3 replicated 2024-09-21T11:53:34.189 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-21T11:53:34.608 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.609 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:34.609 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:34 smithi193 ceph-mon[33333]: Deploying daemon rgw.foo.a on smithi107 2024-09-21T11:53:34.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 -- 172.21.15.193:0/483593283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca41018e0 msgr2=0x7fbca4101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 --2- 172.21.15.193:0/483593283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca41018e0 0x7fbca4101ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009930 tx=0x7fbc9402a420 comp rx=0 tx=0).stop 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 -- 172.21.15.193:0/483593283 shutdown_connections 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 --2- 172.21.15.193:0/483593283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca4103400 0x7fbca4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 --2- 172.21.15.193:0/483593283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4102a40 0x7fbca4102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 --2- 172.21.15.193:0/483593283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca41018e0 0x7fbca4101ce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 -- 172.21.15.193:0/483593283 >> 172.21.15.193:0/483593283 conn(0x7fbca40fd1b0 msgr2=0x7fbca40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 -- 172.21.15.193:0/483593283 shutdown_connections 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 -- 172.21.15.193:0/483593283 wait complete. 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 Processor -- start 2024-09-21T11:53:34.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.693+0000 7fbca9a73640 1 -- start start 2024-09-21T11:53:34.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca9a73640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca41018e0 0x7fbca41056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:34.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca9a73640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca4102a40 0x7fbca4103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:34.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca9a73640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 0x7fbca4104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:34.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca9a73640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbca410c9d0 con 0x7fbca41018e0 2024-09-21T11:53:34.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca9a73640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbca410c850 con 0x7fbca4102a40 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca9a73640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fbca410cb50 con 0x7fbca4103400 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 0x7fbca4104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 0x7fbca4104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.193:58538/0 (socket says 172.21.15.193:58538) 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 -- 172.21.15.193:0/3558302161 learned_addr learned my addr 172.21.15.193:0/3558302161 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca37fe640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca41018e0 0x7fbca41056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbc9bfff640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca4102a40 0x7fbca4103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 -- 172.21.15.193:0/3558302161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca4102a40 msgr2=0x7fbca4103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca4102a40 0x7fbca4103f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 -- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca41018e0 msgr2=0x7fbca41056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca41018e0 0x7fbca41056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.694+0000 7fbca3fff640 1 -- 172.21.15.193:0/3558302161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca4104d10 con 0x7fbca4103400 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.695+0000 7fbca3fff640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 0x7fbca4104450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009fd0 tx=0x7fbc9c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.695+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c017070 con 0x7fbca4103400 2024-09-21T11:53:34.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.695+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca41a2440 con 0x7fbca4103400 2024-09-21T11:53:34.699 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.695+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fbc9c003dc0 con 0x7fbca4103400 2024-09-21T11:53:34.699 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.695+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004800 con 0x7fbca4103400 2024-09-21T11:53:34.699 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.695+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41a2900 con 0x7fbca4103400 2024-09-21T11:53:34.699 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.696+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca4101ce0 con 0x7fbca4103400 2024-09-21T11:53:34.699 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.696+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7fbc9c003970 con 0x7fbca4103400 2024-09-21T11:53:34.700 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.697+0000 7fbca17fa640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbc9007b060 0x7fbc9007d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:34.700 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.697+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 5287+0+0 (secure 0 0 0) 0x7fbc9c09dc80 con 0x7fbca4103400 2024-09-21T11:53:34.700 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.697+0000 7fbca37fe640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbc9007b060 0x7fbc9007d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:34.700 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.698+0000 7fbca37fe640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbc9007b060 0x7fbc9007d520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc940057c0 tx=0x7fbc940058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:34.703 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.702+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbc9c069540 con 0x7fbca4103400 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[30293]: Deploying daemon rgw.foo.a on smithi107 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:34.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:34 smithi107 ceph-mon[41097]: Deploying daemon rgw.foo.a on smithi107 2024-09-21T11:53:34.881 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:34.879+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7fbca402e930 con 0x7fbca4103400 2024-09-21T11:53:35.367 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.366+0000 7fbca17fa640 1 -- 172.21.15.193:0/3558302161 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v53) ==== 160+0+0 (secure 0 0 0) 0x7fbc9c06e3f0 con 0x7fbca4103400 2024-09-21T11:53:35.367 INFO:teuthology.orchestra.run.smithi193.stderr:pool 'datapool' created 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbc9007b060 msgr2=0x7fbc9007d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbc9007b060 0x7fbc9007d520 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc940057c0 tx=0x7fbc940058d0 comp rx=0 tx=0).stop 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 msgr2=0x7fbca4104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 0x7fbca4104450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009fd0 tx=0x7fbc9c00b260 comp rx=0 tx=0).stop 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 shutdown_connections 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fbc9007b060 0x7fbc9007d520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbca4103400 0x7fbca4104450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbca4102a40 0x7fbca4103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 --2- 172.21.15.193:0/3558302161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca41018e0 0x7fbca41056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 >> 172.21.15.193:0/3558302161 conn(0x7fbca40fd1b0 msgr2=0x7fbca406b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 shutdown_connections 2024-09-21T11:53:35.370 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:35.368+0000 7fbca9a73640 1 -- 172.21.15.193:0/3558302161 wait complete. 2024-09-21T11:53:35.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:35 smithi107 ceph-mon[30293]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/3558302161' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:53:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:53:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:35 smithi107 ceph-mon[41097]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/3558302161' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:53:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:53:35.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:35 smithi193 ceph-mon[33333]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:35.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/3558302161' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:53:35.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:53:35.837 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 11:53:35 smithi107 podman[67923]: 2024-09-21 11:53:35.765348338 +0000 UTC m=+0.348584600 container start a24de966ed5b0fc31b2b5652635abf7e7b29d5c1487f49bb53d82080d0d8aec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-21T11:53:35.837 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 11:53:35 smithi107 bash[67923]: a24de966ed5b0fc31b2b5652635abf7e7b29d5c1487f49bb53d82080d0d8aec3 2024-09-21T11:53:36.011 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- rbd pool init datapool 2024-09-21T11:53:36.253 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:36.253 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 11:53:35 smithi107 systemd[1]: Started Ceph rgw.foo.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:53:36.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-21T11:53:36.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: osdmap e53: 8 total, 8 up, 8 in 2024-09-21T11:53:36.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: osdmap e53: 8 total, 8 up, 8 in 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:36 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:36.677 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-21T11:53:36.677 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: osdmap e53: 8 total, 8 up, 8 in 2024-09-21T11:53:36.678 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.678 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.678 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.678 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.678 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:36.678 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:36 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: pgmap v188: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: osdmap e54: 8 total, 8 up, 8 in 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1087524421' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/893316489' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:53:37.561 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[41097]: Checking dashboard <-> RGW credentials 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: pgmap v188: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: osdmap e54: 8 total, 8 up, 8 in 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1087524421' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-21T11:53:37.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/893316489' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:53:37.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:53:37.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:37.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:37.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:37.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:37.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:37 smithi107 ceph-mon[30293]: Checking dashboard <-> RGW credentials 2024-09-21T11:53:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: pgmap v188: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-21T11:53:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:53:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: osdmap e54: 8 total, 8 up, 8 in 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1087524421' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/893316489' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:37 smithi193 ceph-mon[33333]: Checking dashboard <-> RGW credentials 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1087524421' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: osdmap e55: 8 total, 8 up, 8 in 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: pgmap v191: 36 pgs: 8 creating+peering, 2 active+clean, 26 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: Cluster is now healthy 2024-09-21T11:53:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[41097]: osdmap e56: 8 total, 8 up, 8 in 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1087524421' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: osdmap e55: 8 total, 8 up, 8 in 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: pgmap v191: 36 pgs: 8 creating+peering, 2 active+clean, 26 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: Cluster is now healthy 2024-09-21T11:53:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:38 smithi107 ceph-mon[30293]: osdmap e56: 8 total, 8 up, 8 in 2024-09-21T11:53:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1087524421' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-21T11:53:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-21T11:53:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: osdmap e55: 8 total, 8 up, 8 in 2024-09-21T11:53:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: pgmap v191: 36 pgs: 8 creating+peering, 2 active+clean, 26 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:53:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:53:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: Cluster is now healthy 2024-09-21T11:53:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:38 smithi193 ceph-mon[33333]: osdmap e56: 8 total, 8 up, 8 in 2024-09-21T11:53:39.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:39 smithi193 ceph-mon[33333]: osdmap e57: 8 total, 8 up, 8 in 2024-09-21T11:53:39.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[30293]: osdmap e57: 8 total, 8 up, 8 in 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:53:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:39 smithi107 ceph-mon[41097]: osdmap e57: 8 total, 8 up, 8 in 2024-09-21T11:53:40.025 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.193 --placement '1;smithi193=iscsi.a' 2024-09-21T11:53:40.273 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:40.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:40 smithi193 ceph-mon[33333]: pgmap v194: 68 pgs: 24 creating+peering, 12 active+clean, 32 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:53:40.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:40 smithi193 ceph-mon[33333]: osdmap e58: 8 total, 8 up, 8 in 2024-09-21T11:53:40.667 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:40 smithi107 ceph-mon[30293]: pgmap v194: 68 pgs: 24 creating+peering, 12 active+clean, 32 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:53:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:40 smithi107 ceph-mon[30293]: osdmap e58: 8 total, 8 up, 8 in 2024-09-21T11:53:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:40 smithi107 ceph-mon[41097]: pgmap v194: 68 pgs: 24 creating+peering, 12 active+clean, 32 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:53:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:40 smithi107 ceph-mon[41097]: osdmap e58: 8 total, 8 up, 8 in 2024-09-21T11:53:40.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.754+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/3384009568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf54108700 msgr2=0x7fdf5410abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.754+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/3384009568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf54108700 0x7fdf5410abc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ca0 tx=0x7fdf4c02a8c0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/3384009568 shutdown_connections 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/3384009568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf54108700 0x7fdf5410abc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/3384009568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf54105b50 0x7fdf54107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/3384009568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5406b6b0 0x7fdf541055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/3384009568 >> 172.21.15.193:0/3384009568 conn(0x7fdf540fd1b0 msgr2=0x7fdf540ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/3384009568 shutdown_connections 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/3384009568 wait complete. 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 Processor -- start 2024-09-21T11:53:40.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- start start 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 0x7fdf54198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf54105b50 0x7fdf541988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf54108700 0x7fdf54199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410dad0 con 0x7fdf54105b50 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5410d950 con 0x7fdf5406b6b0 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.755+0000 7fdf5a5b1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5410dc50 con 0x7fdf54108700 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 0x7fdf54198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 0x7fdf54198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39708/0 (socket says 172.21.15.193:39708) 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 -- 172.21.15.193:0/920206539 learned_addr learned my addr 172.21.15.193:0/920206539 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf58b4e640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf54108700 0x7fdf54199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf4bfff640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf54105b50 0x7fdf541988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 -- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf54108700 msgr2=0x7fdf54199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:40.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf54108700 0x7fdf54199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 -- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf54105b50 msgr2=0x7fdf541988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf54105b50 0x7fdf541988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 -- 172.21.15.193:0/920206539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf54199dc0 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf53fff640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 0x7fdf54198290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d950 tx=0x7fdf4400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44017030 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5419a0b0 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fdf44004020 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44005440 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.756+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf541a0790 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.757+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf54105fb0 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.758+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7fdf44009a30 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.758+0000 7fdf51ffb640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fdf4007b060 0x7fdf4007d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.758+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6755+0+0 (secure 0 0 0) 0x7fdf4409df50 con 0x7fdf5406b6b0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.758+0000 7fdf4bfff640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fdf4007b060 0x7fdf4007d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:40.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.759+0000 7fdf4bfff640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fdf4007b060 0x7fdf4007d520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005fd0 tx=0x7fdf3c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:40.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.763+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf44064f10 con 0x7fdf5406b6b0 2024-09-21T11:53:40.954 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.952+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.193", "placement": "1;smithi193=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7fdf54063590 con 0x7fdf4007b060 2024-09-21T11:53:40.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.960+0000 7fdf51ffb640 1 -- 172.21.15.193:0/920206539 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7fdf54063590 con 0x7fdf4007b060 2024-09-21T11:53:40.962 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled iscsi.datapool update... 2024-09-21T11:53:40.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.962+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fdf4007b060 msgr2=0x7fdf4007d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:40.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.962+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fdf4007b060 0x7fdf4007d520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005fd0 tx=0x7fdf3c005850 comp rx=0 tx=0).stop 2024-09-21T11:53:40.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.962+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 msgr2=0x7fdf54198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.962+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 0x7fdf54198290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d950 tx=0x7fdf4400de10 comp rx=0 tx=0).stop 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.962+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 shutdown_connections 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fdf4007b060 0x7fdf4007d520 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf54108700 0x7fdf54199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf54105b50 0x7fdf541988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 --2- 172.21.15.193:0/920206539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf5406b6b0 0x7fdf54198290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 >> 172.21.15.193:0/920206539 conn(0x7fdf540fd1b0 msgr2=0x7fdf54103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 shutdown_connections 2024-09-21T11:53:40.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:40.963+0000 7fdf5a5b1640 1 -- 172.21.15.193:0/920206539 wait complete. 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.193", "placement": "1;smithi193=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: Saving service iscsi.datapool spec with placement smithi193=iscsi.a;count:1 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:53:41.532 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:41 smithi193 ceph-mon[33333]: osdmap e59: 8 total, 8 up, 8 in 2024-09-21T11:53:41.559 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-21T11:53:41.559 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:53:41.559 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-21T11:53:41.589 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:53:41.589 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-21T11:53:41.615 DEBUG:teuthology.orchestra.run.smithi193:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@iscsi.iscsi.a.service 2024-09-21T11:53:41.658 INFO:tasks.cephadm:Adding prometheus.a on smithi193 2024-09-21T11:53:41.658 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply prometheus '1;smithi193=a' 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.193", "placement": "1;smithi193=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: Saving service iscsi.datapool spec with placement smithi193=iscsi.a;count:1 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:53:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[30293]: osdmap e59: 8 total, 8 up, 8 in 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.193", "placement": "1;smithi193=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: Saving service iscsi.datapool spec with placement smithi193=iscsi.a;count:1 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:53:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:41 smithi107 ceph-mon[41097]: osdmap e59: 8 total, 8 up, 8 in 2024-09-21T11:53:41.939 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:42.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 -- 172.21.15.193:0/2349393901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a70 msgr2=0x7f8000102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:42.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2349393901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a70 0x7f8000102ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009930 tx=0x7f7fec02a420 comp rx=0 tx=0).stop 2024-09-21T11:53:42.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 -- 172.21.15.193:0/2349393901 shutdown_connections 2024-09-21T11:53:42.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2349393901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103430 0x7f8000109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2349393901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a70 0x7f8000102ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2349393901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000101910 0x7f8000101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 -- 172.21.15.193:0/2349393901 >> 172.21.15.193:0/2349393901 conn(0x7f80000fd200 msgr2=0x7f80000ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:42.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 -- 172.21.15.193:0/2349393901 shutdown_connections 2024-09-21T11:53:42.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.423+0000 7f8006ae4640 1 -- 172.21.15.193:0/2349393901 wait complete. 2024-09-21T11:53:42.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 Processor -- start 2024-09-21T11:53:42.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 -- start start 2024-09-21T11:53:42.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000101910 0x7f8000197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:42.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a70 0x7f8000197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 0x7f80001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f800010cae0 con 0x7f8000101910 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f800010c960 con 0x7f8000103430 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8006ae4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f800010cc60 con 0x7f8000102a70 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 0x7f80001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 0x7f80001986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39728/0 (socket says 172.21.15.193:39728) 2024-09-21T11:53:42.430 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 -- 172.21.15.193:0/2708696364 learned_addr learned my addr 172.21.15.193:0/2708696364 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:42.431 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8004880640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000101910 0x7f8000197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:42.431 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f7ffffff640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a70 0x7f8000197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:42.431 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 -- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a70 msgr2=0x7f8000197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:42.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a70 0x7f8000197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 -- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000101910 msgr2=0x7f8000197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:42.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000101910 0x7f8000197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 -- 172.21.15.193:0/2708696364 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8000198f50 con 0x7f8000103430 2024-09-21T11:53:42.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8004880640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000101910 0x7f8000197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:53:42.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f7ffffff640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a70 0x7f8000197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:53:42.433 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.424+0000 7f8005081640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 0x7f80001986b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b970 tx=0x7f7ff800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:42.433 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.425+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8019070 con 0x7f8000103430 2024-09-21T11:53:42.433 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.425+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7ff8014440 con 0x7f8000103430 2024-09-21T11:53:42.433 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.425+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8014e60 con 0x7f8000103430 2024-09-21T11:53:42.433 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.425+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8000199240 con 0x7f8000103430 2024-09-21T11:53:42.433 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.425+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800019f920 con 0x7f8000103430 2024-09-21T11:53:42.434 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.426+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8000101d90 con 0x7f8000103430 2024-09-21T11:53:42.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.426+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f7ff801c070 con 0x7f8000103430 2024-09-21T11:53:42.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.427+0000 7f7ffdffb640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7ff007b060 0x7f7ff007d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:42.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.427+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 7121+0+0 (secure 0 0 0) 0x7f7ff809da60 con 0x7f8000103430 2024-09-21T11:53:42.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.427+0000 7f8004880640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7ff007b060 0x7f7ff007d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:42.436 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.428+0000 7f8004880640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7ff007b060 0x7f7ff007d520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009930 tx=0x7f7ff4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:42.436 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.431+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ff8068d10 con 0x7f8000103430 2024-09-21T11:53:42.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.622+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}) -- 0x7f800002e7b0 con 0x7f7ff007b060 2024-09-21T11:53:42.624 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:42 smithi193 ceph-mon[33333]: pgmap v196: 100 pgs: 24 creating+peering, 41 active+clean, 35 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-21T11:53:42.624 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:42 smithi193 ceph-mon[33333]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:53:42.624 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:42 smithi193 ceph-mon[33333]: osdmap e60: 8 total, 8 up, 8 in 2024-09-21T11:53:42.624 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.624 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.624 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.629+0000 7f7ffdffb640 1 -- 172.21.15.193:0/2708696364 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (secure 0 0 0) 0x7f800002e7b0 con 0x7f7ff007b060 2024-09-21T11:53:42.631 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled prometheus update... 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7ff007b060 msgr2=0x7f7ff007d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7ff007b060 0x7f7ff007d520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009930 tx=0x7f7ff4008040 comp rx=0 tx=0).stop 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 msgr2=0x7f80001986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 0x7f80001986b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b970 tx=0x7f7ff800be30 comp rx=0 tx=0).stop 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 shutdown_connections 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7ff007b060 0x7f7ff007d520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000103430 0x7f80001986b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a70 0x7f8000197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 --2- 172.21.15.193:0/2708696364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000101910 0x7f8000197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 >> 172.21.15.193:0/2708696364 conn(0x7f80000fd200 msgr2=0x7f80000fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 shutdown_connections 2024-09-21T11:53:42.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:42.631+0000 7f8006ae4640 1 -- 172.21.15.193:0/2708696364 wait complete. 2024-09-21T11:53:42.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[41097]: pgmap v196: 100 pgs: 24 creating+peering, 41 active+clean, 35 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[41097]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[41097]: osdmap e60: 8 total, 8 up, 8 in 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[30293]: pgmap v196: 100 pgs: 24 creating+peering, 41 active+clean, 35 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[30293]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[30293]: osdmap e60: 8 total, 8 up, 8 in 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:53:43.299 DEBUG:teuthology.orchestra.run.smithi193:prometheus.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@prometheus.a.service 2024-09-21T11:53:43.302 INFO:tasks.cephadm:Adding node-exporter.a on smithi107 2024-09-21T11:53:43.302 INFO:tasks.cephadm:Adding node-exporter.b on smithi193 2024-09-21T11:53:43.302 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply node-exporter '2;smithi107=a;smithi193=b' 2024-09-21T11:53:43.593 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:43.984 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: osdmap e61: 8 total, 8 up, 8 in 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:43.985 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: osdmap e61: 8 total, 8 up, 8 in 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: osdmap e61: 8 total, 8 up, 8 in 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4164906608' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:53:44.075 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 -- 172.21.15.193:0/1430639727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018103b60 msgr2=0x7ff018103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 --2- 172.21.15.193:0/1430639727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018103b60 0x7ff018103fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff004009930 tx=0x7ff00402a420 comp rx=0 tx=0).stop 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 -- 172.21.15.193:0/1430639727 shutdown_connections 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 --2- 172.21.15.193:0/1430639727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff018104520 0x7ff01810aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 --2- 172.21.15.193:0/1430639727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018103b60 0x7ff018103fe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 --2- 172.21.15.193:0/1430639727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff018102a00 0x7ff018102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 -- 172.21.15.193:0/1430639727 >> 172.21.15.193:0/1430639727 conn(0x7ff0180fe2f0 msgr2=0x7ff018100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 -- 172.21.15.193:0/1430639727 shutdown_connections 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.064+0000 7ff01f178640 1 -- 172.21.15.193:0/1430639727 wait complete. 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 Processor -- start 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 -- start start 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff018102a00 0x7ff018198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff018103b60 0x7ff018198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:44.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 0x7ff018199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff01810dbd0 con 0x7ff018103b60 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff01810da50 con 0x7ff018104520 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01f178640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff01810dd50 con 0x7ff018102a00 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 0x7ff018199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 0x7ff018199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39750/0 (socket says 172.21.15.193:39750) 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 -- 172.21.15.193:0/2350166173 learned_addr learned my addr 172.21.15.193:0/2350166173 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff017fff640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff018103b60 0x7ff018198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01cf14640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff018102a00 0x7ff018198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 -- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff018102a00 msgr2=0x7ff018198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff018102a00 0x7ff018198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 -- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff018103b60 msgr2=0x7ff018198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff018103b60 0x7ff018198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.065+0000 7ff01d715640 1 -- 172.21.15.193:0/2350166173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff018199fc0 con 0x7ff018104520 2024-09-21T11:53:44.077 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.066+0000 7ff01d715640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 0x7ff018199720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b970 tx=0x7ff01000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.066+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010019070 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.066+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ff010014440 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.066+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01819a2b0 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.066+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100075d0 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.066+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0181a0990 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.067+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff018102e80 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.067+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7ff01001c070 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.068+0000 7ff015ffb640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7ff00807b060 0x7ff00807d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.068+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 7132+0+0 (secure 0 0 0) 0x7ff01009dbd0 con 0x7ff018104520 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.068+0000 7ff01cf14640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7ff00807b060 0x7ff00807d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.069+0000 7ff01cf14640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7ff00807b060 0x7ff00807d520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009990 tx=0x7ff00c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:44.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.072+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff010068e00 con 0x7ff018104520 2024-09-21T11:53:44.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.267+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi193=b", "target": ["mon-mgr", ""]}) -- 0x7ff01802e810 con 0x7ff00807b060 2024-09-21T11:53:44.276 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.274+0000 7ff015ffb640 1 -- 172.21.15.193:0/2350166173 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7ff01802e810 con 0x7ff00807b060 2024-09-21T11:53:44.276 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled node-exporter update... 2024-09-21T11:53:44.278 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7ff00807b060 msgr2=0x7ff00807d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:44.278 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7ff00807b060 0x7ff00807d520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009990 tx=0x7ff00c009110 comp rx=0 tx=0).stop 2024-09-21T11:53:44.278 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 msgr2=0x7ff018199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:44.278 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 0x7ff018199720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b970 tx=0x7ff01000bd70 comp rx=0 tx=0).stop 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 shutdown_connections 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7ff00807b060 0x7ff00807d520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff018104520 0x7ff018199720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff018103b60 0x7ff018198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 --2- 172.21.15.193:0/2350166173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff018102a00 0x7ff018198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.276+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 >> 172.21.15.193:0/2350166173 conn(0x7ff0180fe2f0 msgr2=0x7ff0180fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:44.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.277+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 shutdown_connections 2024-09-21T11:53:44.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:44.277+0000 7ff01f178640 1 -- 172.21.15.193:0/2350166173 wait complete. 2024-09-21T11:53:44.959 DEBUG:teuthology.orchestra.run.smithi107:node-exporter.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.a.service 2024-09-21T11:53:44.962 DEBUG:teuthology.orchestra.run.smithi193:node-exporter.b> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.b.service 2024-09-21T11:53:44.964 INFO:tasks.cephadm:Adding alertmanager.a on smithi107 2024-09-21T11:53:44.964 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply alertmanager '1;smithi107=a' 2024-09-21T11:53:45.500 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: pgmap v199: 132 pgs: 16 creating+peering, 59 active+clean, 57 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: Saving service node-exporter spec with placement smithi107=a;smithi193=b;count:2 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: osdmap e62: 8 total, 8 up, 8 in 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-21T11:53:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-21T11:53:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:45 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:45.751 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: pgmap v199: 132 pgs: 16 creating+peering, 59 active+clean, 57 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: Saving service node-exporter spec with placement smithi107=a;smithi193=b;count:2 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: osdmap e62: 8 total, 8 up, 8 in 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: pgmap v199: 132 pgs: 16 creating+peering, 59 active+clean, 57 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: Saving service node-exporter spec with placement smithi107=a;smithi193=b;count:2 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3918603414' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: osdmap e62: 8 total, 8 up, 8 in 2024-09-21T11:53:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:45.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-21T11:53:45.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-21T11:53:45.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:45 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:46 smithi193 ceph-mon[33333]: Deploying daemon iscsi.iscsi.a on smithi193 2024-09-21T11:53:46.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:46 smithi107 ceph-mon[30293]: Deploying daemon iscsi.iscsi.a on smithi193 2024-09-21T11:53:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:46 smithi107 ceph-mon[41097]: Deploying daemon iscsi.iscsi.a on smithi193 2024-09-21T11:53:46.868 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:46 smithi193 systemd[1]: Starting Ceph iscsi.iscsi.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:53:47.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.954+0000 7f8008064640 1 -- 172.21.15.193:0/1886618039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000104520 msgr2=0x7f800010aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:47.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.954+0000 7f8008064640 1 --2- 172.21.15.193:0/1886618039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000104520 0x7f800010aea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009cd0 tx=0x7f7ff802a8e0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- 172.21.15.193:0/1886618039 shutdown_connections 2024-09-21T11:53:47.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 --2- 172.21.15.193:0/1886618039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000104520 0x7f800010aea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 --2- 172.21.15.193:0/1886618039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103b60 0x7f8000103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 --2- 172.21.15.193:0/1886618039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000102a00 0x7f8000102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- 172.21.15.193:0/1886618039 >> 172.21.15.193:0/1886618039 conn(0x7f80000fe2f0 msgr2=0x7f8000100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- 172.21.15.193:0/1886618039 shutdown_connections 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- 172.21.15.193:0/1886618039 wait complete. 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 Processor -- start 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- start start 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 0x7f80001982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103b60 0x7f8000198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000104520 0x7f8000199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f800010db40 con 0x7f8000103b60 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f800010d9c0 con 0x7f8000102a00 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8008064640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f800010dcc0 con 0x7f8000104520 2024-09-21T11:53:47.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.955+0000 7f8005e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 0x7f80001982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 0x7f80001982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39758/0 (socket says 172.21.15.193:39758) 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 -- 172.21.15.193:0/196907981 learned_addr learned my addr 172.21.15.193:0/196907981 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f80055ff640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103b60 0x7f8000198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8006601640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000104520 0x7f8000199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 -- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000104520 msgr2=0x7f8000199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000104520 0x7f8000199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 -- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103b60 msgr2=0x7f8000198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103b60 0x7f8000198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 -- 172.21.15.193:0/196907981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8000199e10 con 0x7f8000102a00 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8005e00640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 0x7f80001982e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00d770 tx=0x7f7ffc00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc017070 con 0x7f8000102a00 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7ffc005470 con 0x7f8000102a00 2024-09-21T11:53:47.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800019a100 con 0x7f8000102a00 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc004b40 con 0x7f8000102a00 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.956+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80001a07e0 con 0x7f8000102a00 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.957+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8000102e00 con 0x7f8000102a00 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.958+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f7ffc004ce0 con 0x7f8000102a00 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.958+0000 7f7ff6ffd640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7fec07b060 0x7f7fec07d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.958+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 7148+0+0 (secure 0 0 0) 0x7f7ffc09e060 con 0x7f8000102a00 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.959+0000 7f80055ff640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7fec07b060 0x7f7fec07d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.959+0000 7f80055ff640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7fec07b060 0x7f7fec07d520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0005d00 tx=0x7f7ff000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:47.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:46.963+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ffc069280 con 0x7f8000102a00 2024-09-21T11:53:47.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.156+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}) -- 0x7f8000063590 con 0x7f7fec07b060 2024-09-21T11:53:47.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.163+0000 7f7ff6ffd640 1 -- 172.21.15.193:0/196907981 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (secure 0 0 0) 0x7f8000063590 con 0x7f7fec07b060 2024-09-21T11:53:47.165 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled alertmanager update... 2024-09-21T11:53:47.167 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.165+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7fec07b060 msgr2=0x7f7fec07d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.165+0000 7f8008064640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7fec07b060 0x7f7fec07d520 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0005d00 tx=0x7f7ff000b410 comp rx=0 tx=0).stop 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.165+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 msgr2=0x7f80001982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.165+0000 7f8008064640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 0x7f80001982e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00d770 tx=0x7f7ffc00dc30 comp rx=0 tx=0).stop 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 shutdown_connections 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7fec07b060 0x7f7fec07d520 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8000104520 0x7f8000199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8000103b60 0x7f8000198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 --2- 172.21.15.193:0/196907981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8000102a00 0x7f80001982e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 >> 172.21.15.193:0/196907981 conn(0x7f80000fe2f0 msgr2=0x7f80000ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 shutdown_connections 2024-09-21T11:53:47.168 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:47.166+0000 7f8008064640 1 -- 172.21.15.193:0/196907981 wait complete. 2024-09-21T11:53:47.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:47 smithi193 ceph-mon[33333]: pgmap v202: 132 pgs: 126 active+clean, 6 unknown; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 4.7 KiB/s wr, 111 op/s 2024-09-21T11:53:47.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:47 smithi193 ceph-mon[33333]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:53:47.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:47 smithi193 ceph-mon[33333]: Cluster is now healthy 2024-09-21T11:53:47.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:47 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:47.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:47 smithi193 ceph-mon[33333]: osdmap e63: 8 total, 8 up, 8 in 2024-09-21T11:53:47.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[30293]: pgmap v202: 132 pgs: 126 active+clean, 6 unknown; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 4.7 KiB/s wr, 111 op/s 2024-09-21T11:53:47.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[30293]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[30293]: Cluster is now healthy 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[30293]: osdmap e63: 8 total, 8 up, 8 in 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[41097]: pgmap v202: 132 pgs: 126 active+clean, 6 unknown; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 4.7 KiB/s wr, 111 op/s 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[41097]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[41097]: Cluster is now healthy 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:47 smithi107 ceph-mon[41097]: osdmap e63: 8 total, 8 up, 8 in 2024-09-21T11:53:47.791 DEBUG:teuthology.orchestra.run.smithi107:alertmanager.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@alertmanager.a.service 2024-09-21T11:53:47.794 INFO:tasks.cephadm:Adding grafana.a on smithi193 2024-09-21T11:53:47.794 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph orch apply grafana '1;smithi193=a' 2024-09-21T11:53:48.020 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:47 smithi193 podman[60581]: 2024-09-21 11:53:47.805079707 +0000 UTC m=+0.016087199 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T11:53:48.021 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:47 smithi193 podman[60581]: 2024-09-21 11:53:47.921167349 +0000 UTC m=+0.132174821 container create 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2) 2024-09-21T11:53:48.193 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:48.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 podman[60581]: 2024-09-21 11:53:48.081717303 +0000 UTC m=+0.292724791 container init 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T11:53:48.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 podman[60581]: 2024-09-21 11:53:48.086764854 +0000 UTC m=+0.297772327 container start 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T11:53:48.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 bash[60581]: 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae 2024-09-21T11:53:48.353 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 systemd[1]: Started Ceph iscsi.iscsi.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:53:48.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-mon[33333]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:48.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-mon[33333]: Saving service alertmanager spec with placement smithi107=a;count:1 2024-09-21T11:53:48.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.581 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[41097]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:48.581 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[41097]: Saving service alertmanager spec with placement smithi107=a;count:1 2024-09-21T11:53:48.581 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.581 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.581 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[30293]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[30293]: Saving service alertmanager spec with placement smithi107=a;count:1 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.582 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:48 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:48.619 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug Started the configuration object watcher 2024-09-21T11:53:48.619 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug Checking for config object changes every 1s 2024-09-21T11:53:48.619 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug Processing osd blocklist entries for this node 2024-09-21T11:53:48.940 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug Reading the configuration object to update local LIO configuration 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug Configuration does not have an entry for this host(smithi193.front.sepia.ceph.com) - nothing to define to LIO 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: * Environment: production 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: Use a production WSGI server instead. 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: * Debug mode: off 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug * Running on all addresses. 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: * Running on all addresses. 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug * Running on http://[fe80::ec4:7aff:fe88:7253]:5000/ (Press CTRL+C to quit) 2024-09-21T11:53:48.941 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: * Running on http://[fe80::ec4:7aff:fe88:7253]:5000/ (Press CTRL+C to quit) 2024-09-21T11:53:48.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 -- 172.21.15.193:0/921570327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c1018e0 msgr2=0x7f1d3c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 --2- 172.21.15.193:0/921570327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c1018e0 0x7f1d3c101ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009cd0 tx=0x7f1d2c02a8e0 comp rx=0 tx=0).stop 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 -- 172.21.15.193:0/921570327 shutdown_connections 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 --2- 172.21.15.193:0/921570327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d3c103400 0x7f1d3c109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 --2- 172.21.15.193:0/921570327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d3c102a40 0x7f1d3c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 --2- 172.21.15.193:0/921570327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c1018e0 0x7f1d3c101ce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 -- 172.21.15.193:0/921570327 >> 172.21.15.193:0/921570327 conn(0x7f1d3c0fd1b0 msgr2=0x7f1d3c0ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 -- 172.21.15.193:0/921570327 shutdown_connections 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 -- 172.21.15.193:0/921570327 wait complete. 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.938+0000 7f1d419b2640 1 Processor -- start 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 -- start start 2024-09-21T11:53:48.947 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d3c1018e0 0x7f1d3c1056c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 0x7f1d3c103f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d3c103400 0x7f1d3c104470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c10ca30 con 0x7f1d3c103400 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c10c8b0 con 0x7f1d3c102a40 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c10cbb0 con 0x7f1d3c1018e0 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 0x7f1d3c103f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 0x7f1d3c103f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:45356/0 (socket says 172.21.15.193:45356) 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 -- 172.21.15.193:0/1003838580 learned_addr learned my addr 172.21.15.193:0/1003838580 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3affd640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d3c1018e0 0x7f1d3c1056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3b7fe640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d3c103400 0x7f1d3c104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 -- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d3c1018e0 msgr2=0x7f1d3c1056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d3c1018e0 0x7f1d3c1056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 -- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d3c103400 msgr2=0x7f1d3c104470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:48.948 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d3c103400 0x7f1d3c104470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 -- 172.21.15.193:0/1003838580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c1049b0 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d3a7fc640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 0x7f1d3c103f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b4a0 tx=0x7f1d3000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30014070 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c104ca0 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.939+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f1d30010a60 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.940+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300179e0 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.940+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c1a6ec0 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.941+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c101ce0 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.941+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 103234+0+0 (secure 0 0 0) 0x7f1d30017070 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.942+0000 7f1d409b0640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f1d2407b060 0x7f1d2407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.942+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f1d3009d810 con 0x7f1d3c102a40 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.942+0000 7f1d3affd640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f1d2407b060 0x7f1d2407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:48.949 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.942+0000 7f1d3affd640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f1d2407b060 0x7f1d2407d520 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c006010 tx=0x7f1d2c009900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:48.950 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:48.946+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d30068ab0 con 0x7f1d3c102a40 2024-09-21T11:53:49.146 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.144+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 --> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}) -- 0x7f1d3c02e7d0 con 0x7f1d2407b060 2024-09-21T11:53:49.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.151+0000 7f1d409b0640 1 -- 172.21.15.193:0/1003838580 <== mgr.14152 v2:172.21.15.107:6800/2130999972 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1d3c02e7d0 con 0x7f1d2407b060 2024-09-21T11:53:49.153 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled grafana update... 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f1d2407b060 msgr2=0x7f1d2407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f1d2407b060 0x7f1d2407d520 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c006010 tx=0x7f1d2c009900 comp rx=0 tx=0).stop 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 msgr2=0x7f1d3c103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 0x7f1d3c103f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b4a0 tx=0x7f1d3000b960 comp rx=0 tx=0).stop 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 shutdown_connections 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f1d2407b060 0x7f1d2407d520 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d3c103400 0x7f1d3c104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:49.156 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d3c102a40 0x7f1d3c103f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 --2- 172.21.15.193:0/1003838580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d3c1018e0 0x7f1d3c1056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 >> 172.21.15.193:0/1003838580 conn(0x7f1d3c0fd1b0 msgr2=0x7f1d3c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 shutdown_connections 2024-09-21T11:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:49.154+0000 7f1d419b2640 1 -- 172.21.15.193:0/1003838580 wait complete. 2024-09-21T11:53:49.543 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:49 smithi193 ceph-mon[33333]: pgmap v204: 132 pgs: 132 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.8 KiB/s wr, 110 op/s 2024-09-21T11:53:49.544 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:49 smithi193 ceph-mon[33333]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-21T11:53:49.544 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/3312012188' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T11:53:49.544 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:49 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[30293]: pgmap v204: 132 pgs: 132 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.8 KiB/s wr, 110 op/s 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[30293]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/3312012188' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[41097]: pgmap v204: 132 pgs: 132 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.8 KiB/s wr, 110 op/s 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[41097]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/3312012188' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T11:53:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:49 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:49.828 DEBUG:teuthology.orchestra.run.smithi193:grafana.a> sudo journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@grafana.a.service 2024-09-21T11:53:49.831 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T11:53:49.831 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T11:53:50.088 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:50.476 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:50 smithi107 ceph-mon[41097]: Deploying daemon prometheus.a on smithi193 2024-09-21T11:53:50.476 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:50 smithi107 ceph-mon[41097]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:50.476 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:50 smithi107 ceph-mon[41097]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-21T11:53:50.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:50 smithi107 ceph-mon[30293]: Deploying daemon prometheus.a on smithi193 2024-09-21T11:53:50.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:50 smithi107 ceph-mon[30293]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:50.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:50 smithi107 ceph-mon[30293]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-21T11:53:50.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 -- 172.21.15.107:0/2419510850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888103440 msgr2=0x7f4888109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:50.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 --2- 172.21.15.107:0/2419510850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888103440 0x7f4888109dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4880009c40 tx=0x7f488002a710 comp rx=0 tx=0).stop 2024-09-21T11:53:50.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 -- 172.21.15.107:0/2419510850 shutdown_connections 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 --2- 172.21.15.107:0/2419510850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888103440 0x7f4888109dc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 --2- 172.21.15.107:0/2419510850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4888102a80 0x7f4888102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 --2- 172.21.15.107:0/2419510850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f4888101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.565+0000 7f488e228640 1 -- 172.21.15.107:0/2419510850 >> 172.21.15.107:0/2419510850 conn(0x7f48880fd1d0 msgr2=0x7f48880ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 -- 172.21.15.107:0/2419510850 shutdown_connections 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 -- 172.21.15.107:0/2419510850 wait complete. 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 Processor -- start 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 -- start start 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f48881983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888102a80 0x7f4888198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4888103440 0x7f4888199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:50.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f488810ca90 con 0x7f4888101920 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f488810c910 con 0x7f4888103440 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.566+0000 7f488e228640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f488810cc10 con 0x7f4888102a80 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f48881983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f48881983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35644/0 (socket says 172.21.15.107:35644) 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 -- 172.21.15.107:0/76714133 learned_addr learned my addr 172.21.15.107:0/76714133 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f4886ffd640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888102a80 0x7f4888198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f4887fff640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4888103440 0x7f4888199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 -- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888102a80 msgr2=0x7f4888198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888102a80 0x7f4888198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 -- 172.21.15.107:0/76714133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4888103440 msgr2=0x7f4888199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4888103440 0x7f4888199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 -- 172.21.15.107:0/76714133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4888199f20 con 0x7f4888101920 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f48877fe640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f48881983f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9c0 tx=0x7f487800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:50.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487800cd80 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f4878004470 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488819a210 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878003d90 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.567+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48881a08f0 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.568+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888101d20 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.569+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 103285+0+0 (secure 0 0 0) 0x7f4878004000 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.569+0000 7f4884ff9640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f487007b0f0 0x7f487007d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.569+0000 7f4886ffd640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f487007b0f0 0x7f487007d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.569+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f487809eab0 con 0x7f4888101920 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.570+0000 7f4886ffd640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f487007b0f0 0x7f487007d5b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f487c009990 tx=0x7f487c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:50.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.574+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f48780657f0 con 0x7f4888101920 2024-09-21T11:53:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:50 smithi193 ceph-mon[33333]: Deploying daemon prometheus.a on smithi193 2024-09-21T11:53:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:50 smithi193 ceph-mon[33333]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:53:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:50 smithi193 ceph-mon[33333]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-21T11:53:50.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.822+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f488802e7f0 con 0x7f4888101920 2024-09-21T11:53:50.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.826+0000 7f4884ff9640 1 -- 172.21.15.107:0/76714133 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7f4878069bd0 con 0x7f4888101920 2024-09-21T11:53:50.828 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-09-21T11:53:50.828 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDOs+5mHIgZMRAAox6+MaNzOfrd8jk1gzQ/vw== 2024-09-21T11:53:50.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f487007b0f0 msgr2=0x7f487007d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:50.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f487007b0f0 0x7f487007d5b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f487c009990 tx=0x7f487c009110 comp rx=0 tx=0).stop 2024-09-21T11:53:50.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 msgr2=0x7f48881983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:50.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f48881983f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f487800e9c0 tx=0x7f487800edc0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 shutdown_connections 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f487007b0f0 0x7f487007d5b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4888103440 0x7f4888199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4888102a80 0x7f4888198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 --2- 172.21.15.107:0/76714133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4888101920 0x7f48881983f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 >> 172.21.15.107:0/76714133 conn(0x7f48880fd1d0 msgr2=0x7f488806bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.829+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 shutdown_connections 2024-09-21T11:53:50.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:50.830+0000 7f488e228640 1 -- 172.21.15.107:0/76714133 wait complete. 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[30293]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 5.5 KiB/s wr, 175 op/s 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[30293]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/76714133' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/76714133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[41097]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 5.5 KiB/s wr, 175 op/s 2024-09-21T11:53:51.464 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[41097]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-21T11:53:51.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:51.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/76714133' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:51.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/76714133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:53:51.491 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:53:51.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T11:53:51.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T11:53:51.528 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T11:53:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:51 smithi193 ceph-mon[33333]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 5.5 KiB/s wr, 175 op/s 2024-09-21T11:53:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:51 smithi193 ceph-mon[33333]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-21T11:53:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:51 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:51.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/76714133' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:51.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/76714133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:53:51.979 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:52.632 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 -- 172.21.15.193:0/2472249888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 msgr2=0x7f7d4810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/2472249888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d4810aed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009cd0 tx=0x7f7d4002a8e0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 -- 172.21.15.193:0/2472249888 shutdown_connections 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/2472249888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d4810aed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/2472249888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103b90 0x7f7d48104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/2472249888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d48102a30 0x7f7d48102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 -- 172.21.15.193:0/2472249888 >> 172.21.15.193:0/2472249888 conn(0x7f7d480fe320 msgr2=0x7f7d48100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 -- 172.21.15.193:0/2472249888 shutdown_connections 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.565+0000 7f7d4f81e640 1 -- 172.21.15.193:0/2472249888 wait complete. 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 Processor -- start 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 -- start start 2024-09-21T11:53:52.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d48102a30 0x7f7d481982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103b90 0x7f7d48198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d48199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4810db20 con 0x7f7d48103b90 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4810d9a0 con 0x7f7d48104550 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4f81e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4810dca0 con 0x7f7d48102a30 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d48199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d48199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:45372/0 (socket says 172.21.15.193:45372) 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 -- 172.21.15.193:0/4084640269 learned_addr learned my addr 172.21.15.193:0/4084640269 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4cdb9640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103b90 0x7f7d48198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4d5ba640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d48102a30 0x7f7d481982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 -- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d48102a30 msgr2=0x7f7d481982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d48102a30 0x7f7d481982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 -- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103b90 msgr2=0x7f7d48198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:52.634 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103b90 0x7f7d48198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.566+0000 7f7d4ddbb640 1 -- 172.21.15.193:0/4084640269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48199e20 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.567+0000 7f7d4ddbb640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d48199580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009ca0 tx=0x7f7d40003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.567+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40004250 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.567+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7d40004840 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.567+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4003b400 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.567+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4819a0b0 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.567+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d481a07f0 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.568+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d48102e30 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.569+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 103285+0+0 (secure 0 0 0) 0x7f7d40030730 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.569+0000 7f7d3e7fc640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7d3407b0f0 0x7f7d3407d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.569+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f7d4002e030 con 0x7f7d48104550 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.570+0000 7f7d4d5ba640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7d3407b0f0 0x7f7d3407d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:52.635 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.570+0000 7f7d4d5ba640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7d3407b0f0 0x7f7d3407d5b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d38005d00 tx=0x7f7d3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:52.636 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.574+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7d4008a970 con 0x7f7d48104550 2024-09-21T11:53:52.840 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.838+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7d4802e830 con 0x7f7d48104550 2024-09-21T11:53:52.845 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.843+0000 7f7d3e7fc640 1 -- 172.21.15.193:0/4084640269 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7f7d4008f820 con 0x7f7d48104550 2024-09-21T11:53:52.845 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-09-21T11:53:52.845 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQDQs+5mFewXMhAAuDNOL6h3uxUE3wIh5ZpAFA== 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7d3407b0f0 msgr2=0x7f7d3407d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7d3407b0f0 0x7f7d3407d5b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d38005d00 tx=0x7f7d3800b410 comp rx=0 tx=0).stop 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 msgr2=0x7f7d48199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d48199580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009ca0 tx=0x7f7d40003c10 comp rx=0 tx=0).stop 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 shutdown_connections 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7d3407b0f0 0x7f7d3407d5b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d48104550 0x7f7d48199580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d48103b90 0x7f7d48198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 --2- 172.21.15.193:0/4084640269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d48102a30 0x7f7d481982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 >> 172.21.15.193:0/4084640269 conn(0x7f7d480fe320 msgr2=0x7f7d480fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 shutdown_connections 2024-09-21T11:53:52.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T11:53:52.846+0000 7f7d4f81e640 1 -- 172.21.15.193:0/4084640269 wait complete. 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[30293]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.193:0/4084640269' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[41097]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.193:0/4084640269' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:53:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:53 smithi193 ceph-mon[33333]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-09-21T11:53:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.193:0/4084640269' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:53:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:53:54.606 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T11:53:54.606 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-21T11:53:54.606 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T11:53:54.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T11:53:54.900 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 -- 172.21.15.107:0/2478123078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 msgr2=0x7f7b74102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2478123078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b74102040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a410 comp rx=0 tx=0).stop 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 -- 172.21.15.107:0/2478123078 shutdown_connections 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2478123078 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107ff0 0x7f7b7410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2478123078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b74107670 0x7f7b74107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2478123078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b74102040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.398+0000 7f7b79a22640 1 -- 172.21.15.107:0/2478123078 >> 172.21.15.107:0/2478123078 conn(0x7f7b740fd6d0 msgr2=0x7f7b740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 -- 172.21.15.107:0/2478123078 shutdown_connections 2024-09-21T11:53:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 -- 172.21.15.107:0/2478123078 wait complete. 2024-09-21T11:53:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 Processor -- start 2024-09-21T11:53:55.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 -- start start 2024-09-21T11:53:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b741976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107670 0x7f7b74197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.399+0000 7f7b79a22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b74107ff0 0x7f7b7419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b79a22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74111180 con 0x7f7b74101c60 2024-09-21T11:53:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b79a22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74111000 con 0x7f7b74107ff0 2024-09-21T11:53:55.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b79a22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7b74111300 con 0x7f7b74107670 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107670 0x7f7b74197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b72ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b741976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b72ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b741976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57674/0 (socket says 172.21.15.107:57674) 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b72ffd640 1 -- 172.21.15.107:0/2700472552 learned_addr learned my addr 172.21.15.107:0/2700472552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b737fe640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b74107ff0 0x7f7b7419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 -- 172.21.15.107:0/2700472552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b74107ff0 msgr2=0x7f7b7419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b74107ff0 0x7f7b7419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 -- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 msgr2=0x7f7b741976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b741976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 -- 172.21.15.107:0/2700472552 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7419d760 con 0x7f7b74107670 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b72ffd640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b741976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b6a7fc640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107670 0x7f7b74197bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e9f0 tx=0x7f7b5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00ccc0 con 0x7f7b74107670 2024-09-21T11:53:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7419da50 con 0x7f7b74107670 2024-09-21T11:53:55.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7b5c010880 con 0x7f7b74107670 2024-09-21T11:53:55.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c018620 con 0x7f7b74107670 2024-09-21T11:53:55.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.400+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a42e0 con 0x7f7b74107670 2024-09-21T11:53:55.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.401+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b74102f10 con 0x7f7b74107670 2024-09-21T11:53:55.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.402+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 17) ==== 103285+0+0 (secure 0 0 0) 0x7f7b5c010430 con 0x7f7b74107670 2024-09-21T11:53:55.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.402+0000 7f7b78a20640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7b6007b0f0 0x7f7b6007d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:55.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.402+0000 7f7b72ffd640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7b6007b0f0 0x7f7b6007d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:55.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.403+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f7b5c09ea30 con 0x7f7b74107670 2024-09-21T11:53:55.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.403+0000 7f7b72ffd640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7b6007b0f0 0x7f7b6007d5b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b640058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.407+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b5c069b50 con 0x7f7b74107670 2024-09-21T11:53:55.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.585+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f7b741987e0 con 0x7f7b74107670 2024-09-21T11:53:55.589 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:55 smithi107 ceph-mon[41097]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 4.0 KiB/s wr, 125 op/s 2024-09-21T11:53:55.589 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:55 smithi107 ceph-mon[30293]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 4.0 KiB/s wr, 125 op/s 2024-09-21T11:53:55.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.586+0000 7f7b78a20640 1 -- 172.21.15.107:0/2700472552 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v25) ==== 86+0+187 (secure 0 0 0) 0x7f7b5c06ea00 con 0x7f7b74107670 2024-09-21T11:53:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.588+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7b6007b0f0 msgr2=0x7f7b6007d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.588+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7b6007b0f0 0x7f7b6007d5b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b640058d0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.588+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107670 msgr2=0x7f7b74197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.588+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107670 0x7f7b74197bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e9f0 tx=0x7f7b5c00eeb0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 shutdown_connections 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f7b6007b0f0 0x7f7b6007d5b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b74107ff0 0x7f7b7419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b74107670 0x7f7b74197bf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 --2- 172.21.15.107:0/2700472552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b74101c60 0x7f7b741976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 >> 172.21.15.107:0/2700472552 conn(0x7f7b740fd6d0 msgr2=0x7f7b740ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 shutdown_connections 2024-09-21T11:53:55.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:55.589+0000 7f7b79a22640 1 -- 172.21.15.107:0/2700472552 wait complete. 2024-09-21T11:53:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:55 smithi193 ceph-mon[33333]: pgmap v207: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s rd, 4.0 KiB/s wr, 125 op/s 2024-09-21T11:53:56.329 INFO:teuthology.orchestra.run.smithi107.stdout:[{"version":25,"timestamp":"2024-09-21T11:53:44.601587+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"YAZqecbkEOQpqzeZc9r165bVUIJhAZdgr78H1FsB"}]}] 2024-09-21T11:53:56.329 INFO:tasks.ceph_manager:config epoch is 25 2024-09-21T11:53:56.329 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T11:53:56.329 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T11:53:56.330 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T11:53:56.573 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:56.694 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2700472552' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T11:53:56.694 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:56 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:56.694 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2700472552' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T11:53:56.694 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:56 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:56.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2700472552' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T11:53:56.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:56 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:53:57.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 -- 172.21.15.107:0/3572092387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08104540 msgr2=0x7fab0810cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:57.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 --2- 172.21.15.107:0/3572092387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08104540 0x7fab0810cb20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009860 tx=0x7faaf402a340 comp rx=0 tx=0).stop 2024-09-21T11:53:57.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 -- 172.21.15.107:0/3572092387 shutdown_connections 2024-09-21T11:53:57.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 --2- 172.21.15.107:0/3572092387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab0810d0f0 0x7fab0810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 --2- 172.21.15.107:0/3572092387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08104540 0x7fab0810cb20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 --2- 172.21.15.107:0/3572092387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab08103b40 0x7fab08103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 -- 172.21.15.107:0/3572092387 >> 172.21.15.107:0/3572092387 conn(0x7fab080fd6d0 msgr2=0x7fab080ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 -- 172.21.15.107:0/3572092387 shutdown_connections 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.996+0000 7fab0ee43640 1 -- 172.21.15.107:0/3572092387 wait complete. 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 Processor -- start 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 -- start start 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08103b40 0x7fab081988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab08104540 0x7fab08198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 0x7fab0819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fab08112200 con 0x7fab08104540 2024-09-21T11:53:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab08112080 con 0x7fab0810d0f0 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0ee43640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fab08112380 con 0x7fab08103b40 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0d3e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 0x7fab0819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0d3e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 0x7fab0819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:42938/0 (socket says 172.21.15.107:42938) 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0d3e0640 1 -- 172.21.15.107:0/4128100202 learned_addr learned my addr 172.21.15.107:0/4128100202 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab07fff640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab08104540 0x7fab08198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab0cbdf640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08103b40 0x7fab081988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab07fff640 1 -- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08103b40 msgr2=0x7fab081988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab07fff640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08103b40 0x7fab081988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab07fff640 1 -- 172.21.15.107:0/4128100202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 msgr2=0x7fab0819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:57.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab07fff640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 0x7fab0819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.997+0000 7fab07fff640 1 -- 172.21.15.107:0/4128100202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0819e950 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab0d3e0640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 0x7fab0819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab07fff640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab08104540 0x7fab08198de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faaf4035040 tx=0x7faaf4033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf403f020 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0819ebe0 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7faaf40306b0 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4037040 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.998+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081a5300 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:56.999+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab08103f20 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.000+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 103285+0+0 (secure 0 0 0) 0x7faaf4042020 con 0x7fab08104540 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.000+0000 7fab05ffb640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faaf807b0f0 0x7faaf807d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.000+0000 7fab0cbdf640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faaf807b0f0 0x7faaf807d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:57.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.000+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7faaf40be410 con 0x7fab08104540 2024-09-21T11:53:57.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.001+0000 7fab0cbdf640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faaf807b0f0 0x7faaf807d5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faafc0058c0 tx=0x7faafc00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:57.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.004+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faaf4049020 con 0x7fab08104540 2024-09-21T11:53:57.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.230+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fab081a54d0 con 0x7fab08104540 2024-09-21T11:53:57.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.233+0000 7fab05ffb640 1 -- 172.21.15.107:0/4128100202 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+198171 (secure 0 0 0) 0x7faaf4089530 con 0x7fab08104540 2024-09-21T11:53:57.239 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:53:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faaf807b0f0 msgr2=0x7faaf807d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faaf807b0f0 0x7faaf807d5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faafc0058c0 tx=0x7faafc00a250 comp rx=0 tx=0).stop 2024-09-21T11:53:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab08104540 msgr2=0x7fab08198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab08104540 0x7fab08198de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faaf4035040 tx=0x7faaf4033d00 comp rx=0 tx=0).stop 2024-09-21T11:53:57.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 shutdown_connections 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7faaf807b0f0 0x7faaf807d5b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab0810d0f0 0x7fab0819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab08104540 0x7fab08198de0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 --2- 172.21.15.107:0/4128100202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab08103b40 0x7fab081988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 >> 172.21.15.107:0/4128100202 conn(0x7fab080fd6d0 msgr2=0x7fab08104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 shutdown_connections 2024-09-21T11:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:57.237+0000 7fab0ee43640 1 -- 172.21.15.107:0/4128100202 wait complete. 2024-09-21T11:53:57.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:57 smithi107 ceph-mon[41097]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 2.0 KiB/s wr, 70 op/s 2024-09-21T11:53:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4128100202' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T11:53:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:57 smithi107 ceph-mon[30293]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 2.0 KiB/s wr, 70 op/s 2024-09-21T11:53:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4128100202' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T11:53:57.627 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:57 smithi193 ceph-mon[33333]: pgmap v208: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 2.0 KiB/s wr, 70 op/s 2024-09-21T11:53:57.627 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4128100202' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T11:53:57.939 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2130999972},{"type":"v1","addr":"172.21.15.107:6801","nonce":2130999972}]},"active_addr":"172.21.15.107:6801/2130999972","active_change":"2024-09-21T11:48:45.320108+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1661044187}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1505731212}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1010058641}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3337655267}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1353233379}]}]} 2024-09-21T11:53:57.943 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T11:53:57.943 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T11:53:57.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T11:53:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:58.551 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:53:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 -- 172.21.15.107:0/2352057498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 msgr2=0x7f648410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 --2- 172.21.15.107:0/2352057498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f648410e810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f647c009ca0 tx=0x7f647c02a8c0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 -- 172.21.15.107:0/2352057498 shutdown_connections 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 --2- 172.21.15.107:0/2352057498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f648410e810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 --2- 172.21.15.107:0/2352057498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 0x7f6484107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 --2- 172.21.15.107:0/2352057498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6484101c80 0x7f6484102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 -- 172.21.15.107:0/2352057498 >> 172.21.15.107:0/2352057498 conn(0x7f64840fd730 msgr2=0x7f64840ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 -- 172.21.15.107:0/2352057498 shutdown_connections 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.628+0000 7f6489433640 1 -- 172.21.15.107:0/2352057498 wait complete. 2024-09-21T11:53:58.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 Processor -- start 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 -- start start 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6484101c80 0x7f6484199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 0x7f6484199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f6484199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f64841111a0 con 0x7f6484108010 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6484111020 con 0x7f6484101c80 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f6489433640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f6484111320 con 0x7f6484107690 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.629+0000 7f64837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f6484199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f6484199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57688/0 (socket says 172.21.15.107:57688) 2024-09-21T11:53:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 -- 172.21.15.107:0/2678846933 learned_addr learned my addr 172.21.15.107:0/2678846933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f647a7fc640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 0x7f6484199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f6482ffd640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6484101c80 0x7f6484199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 -- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 msgr2=0x7f6484199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 0x7f6484199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 -- 172.21.15.107:0/2678846933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6484101c80 msgr2=0x7f6484199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6484101c80 0x7f6484199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 -- 172.21.15.107:0/2678846933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841a4910 con 0x7f6484108010 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f647a7fc640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 0x7f6484199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f64837fe640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f6484199fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f647c009bf0 tx=0x7f647c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c004040 con 0x7f6484108010 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f647c040070 con 0x7f6484108010 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6484197300 con 0x7f6484108010 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c030af0 con 0x7f6484108010 2024-09-21T11:53:58.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.630+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6484197890 con 0x7f6484108010 2024-09-21T11:53:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.631+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6484102060 con 0x7f6484108010 2024-09-21T11:53:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.632+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 103285+0+0 (secure 0 0 0) 0x7f647c030c90 con 0x7f6484108010 2024-09-21T11:53:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.632+0000 7f6480ff9640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f647007b0f0 0x7f647007d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:53:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.633+0000 7f6482ffd640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f647007b0f0 0x7f647007d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:53:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.633+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f647c0bf800 con 0x7f6484108010 2024-09-21T11:53:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.633+0000 7f6482ffd640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f647007b0f0 0x7f647007d5b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64740058c0 tx=0x7f6474010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:53:58.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.637+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f647c08aab0 con 0x7f6484108010 2024-09-21T11:53:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.817+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6484109df0 con 0x7f6484108010 2024-09-21T11:53:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.818+0000 7f6480ff9640 1 -- 172.21.15.107:0/2678846933 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+22821 (secure 0 0 0) 0x7f647c08f960 con 0x7f6484108010 2024-09-21T11:53:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:53:58.821 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":63,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","created":"2024-09-21T11:47:57.300122+0000","modified":"2024-09-21T11:53:47.211709+0000","last_up_change":"2024-09-21T11:53:27.207544+0000","last_in_change":"2024-09-21T11:53:04.078828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:51:05.622296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-21T11:53:34.884714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-21T11:53:35.804433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-21T11:53:37.455475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-21T11:53:39.397850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-21T11:53:41.458869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e4feb96-c3c7-4f2d-9ba3-1db963213296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6803","nonce":2032034052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6805","nonce":2032034052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6809","nonce":2032034052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6807","nonce":2032034052}]},"public_addr":"172.21.15.107:6803/2032034052","cluster_addr":"172.21.15.107:6805/2032034052","heartbeat_back_addr":"172.21.15.107:6809/2032034052","heartbeat_front_addr":"172.21.15.107:6807/2032034052","state":["exists","up"]},{"osd":1,"uuid":"a464ccec-f1a1-48ae-b233-89519142c774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6811","nonce":3491646662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6813","nonce":3491646662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6817","nonce":3491646662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6815","nonce":3491646662}]},"public_addr":"172.21.15.107:6811/3491646662","cluster_addr":"172.21.15.107:6813/3491646662","heartbeat_back_addr":"172.21.15.107:6817/3491646662","heartbeat_front_addr":"172.21.15.107:6815/3491646662","state":["exists","up"]},{"osd":2,"uuid":"74584ac0-8833-4c1e-9f98-2bef0780286e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6819","nonce":216243970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6821","nonce":216243970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6825","nonce":216243970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6823","nonce":216243970}]},"public_addr":"172.21.15.107:6819/216243970","cluster_addr":"172.21.15.107:6821/216243970","heartbeat_back_addr":"172.21.15.107:6825/216243970","heartbeat_front_addr":"172.21.15.107:6823/216243970","state":["exists","up"]},{"osd":3,"uuid":"661db9ba-cabf-4159-b051-379aa09acc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6827","nonce":1625163749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6829","nonce":1625163749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6833","nonce":1625163749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6831","nonce":1625163749}]},"public_addr":"172.21.15.107:6827/1625163749","cluster_addr":"172.21.15.107:6829/1625163749","heartbeat_back_addr":"172.21.15.107:6833/1625163749","heartbeat_front_addr":"172.21.15.107:6831/1625163749","state":["exists","up"]},{"osd":4,"uuid":"6c71418f-c007-4428-b338-3ebfc82f779d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6801","nonce":3995503105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6803","nonce":3995503105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6807","nonce":3995503105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6805","nonce":3995503105}]},"public_addr":"172.21.15.193:6801/3995503105","cluster_addr":"172.21.15.193:6803/3995503105","heartbeat_back_addr":"172.21.15.193:6807/3995503105","heartbeat_front_addr":"172.21.15.193:6805/3995503105","state":["exists","up"]},{"osd":5,"uuid":"f2576976-ac0c-45e0-8353-95220342328d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6809","nonce":102264086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6811","nonce":102264086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6815","nonce":102264086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6813","nonce":102264086}]},"public_addr":"172.21.15.193:6809/102264086","cluster_addr":"172.21.15.193:6811/102264086","heartbeat_back_addr":"172.21.15.193:6815/102264086","heartbeat_front_addr":"172.21.15.193:6813/102264086","state":["exists","up"]},{"osd":6,"uuid":"14c736d5-64dc-41f2-9cc8-6b8adaba0ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6817","nonce":1896699774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6819","nonce":1896699774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6823","nonce":1896699774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6821","nonce":1896699774}]},"public_addr":"172.21.15.193:6817/1896699774","cluster_addr":"172.21.15.193:6819/1896699774","heartbeat_back_addr":"172.21.15.193:6823/1896699774","heartbeat_front_addr":"172.21.15.193:6821/1896699774","state":["exists","up"]},{"osd":7,"uuid":"fb16cb03-fe65-4189-818c-29a99c0db388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6825","nonce":338565813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6827","nonce":338565813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6831","nonce":338565813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6829","nonce":338565813}]},"public_addr":"172.21.15.193:6825/338565813","cluster_addr":"172.21.15.193:6827/338565813","heartbeat_back_addr":"172.21.15.193:6831/338565813","heartbeat_front_addr":"172.21.15.193:6829/338565813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:50:02.844467+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:50:32.631011+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:02.098307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:31.483117+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:58.119895+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:52:27.691211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:52:56.093529+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:53:25.370332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3723772931":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6800/1807990149":"2024-09-22T11:48:45.320004+0000","172.21.15.107:0/1627664470":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6800/1943709907":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/3703659764":"2024-09-22T11:48:45.320004+0000","172.21.15.107:0/545710521":"2024-09-22T11:48:19.247117+0000","172.21.15.107:6801/1807990149":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6801/1943709907":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/2867069233":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/1351066913":"2024-09-22T11:48:19.247117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T11:53:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f647007b0f0 msgr2=0x7f647007d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f647007b0f0 0x7f647007d5b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64740058c0 tx=0x7f6474010040 comp rx=0 tx=0).stop 2024-09-21T11:53:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 msgr2=0x7f6484199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:53:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f6484199fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f647c009bf0 tx=0x7f647c003ba0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 shutdown_connections 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f647007b0f0 0x7f647007d5b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6484108010 0x7f6484199fd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6484107690 0x7f6484199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 --2- 172.21.15.107:0/2678846933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6484101c80 0x7f6484199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 >> 172.21.15.107:0/2678846933 conn(0x7f64840fd730 msgr2=0x7f64840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 shutdown_connections 2024-09-21T11:53:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:53:58.821+0000 7f6489433640 1 -- 172.21.15.107:0/2678846933 wait complete. 2024-09-21T11:53:59.518 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T11:53:59.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T11:53:59.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:59 smithi107 ceph-mon[30293]: pgmap v209: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 1.8 KiB/s wr, 58 op/s 2024-09-21T11:53:59.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:53:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2678846933' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:53:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:59 smithi107 ceph-mon[41097]: pgmap v209: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 1.8 KiB/s wr, 58 op/s 2024-09-21T11:53:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:53:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2678846933' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:53:59.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:59 smithi193 ceph-mon[33333]: pgmap v209: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 1.8 KiB/s wr, 58 op/s 2024-09-21T11:53:59.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:53:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2678846933' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:53:59.762 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2108888853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac113b10 msgr2=0x7f5fac11a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2108888853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac113b10 0x7f5fac11a310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009ca0 tx=0x7f5fa402a8c0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2108888853 shutdown_connections 2024-09-21T11:54:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2108888853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac113b10 0x7f5fac11a310 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2108888853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 0x7f5fac1135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2108888853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fac10d780 0x7f5fac10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2108888853 >> 172.21.15.107:0/2108888853 conn(0x7f5fac06dc80 msgr2=0x7f5fac06e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2108888853 shutdown_connections 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.205+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2108888853 wait complete. 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 Processor -- start 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 -- start start 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 0x7f5fac1a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 0x7f5fac1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fac113b10 0x7f5fac1a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac11cc00 con 0x7f5fac10d780 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac11ca80 con 0x7f5fac113b10 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb3e62640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac11cd80 con 0x7f5fac113190 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb1bfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 0x7f5fac1a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb1bfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 0x7f5fac1a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57706/0 (socket says 172.21.15.107:57706) 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb1bfe640 1 -- 172.21.15.107:0/2003431568 learned_addr learned my addr 172.21.15.107:0/2003431568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb13fd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 0x7f5fac1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.206+0000 7f5fb23ff640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fac113b10 0x7f5fac1a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb13fd640 1 -- 172.21.15.107:0/2003431568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fac113b10 msgr2=0x7f5fac1a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb13fd640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fac113b10 0x7f5fac1a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb13fd640 1 -- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 msgr2=0x7f5fac1a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb13fd640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 0x7f5fac1a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb13fd640 1 -- 172.21.15.107:0/2003431568 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac1a91f0 con 0x7f5fac113190 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb1bfe640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 0x7f5fac1a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb13fd640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 0x7f5fac1a3680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e9f0 tx=0x7f5f9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00cd80 con 0x7f5fac113190 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f5f9c004470 con 0x7f5fac113190 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1a94e0 con 0x7f5fac113190 2024-09-21T11:54:00.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c003e10 con 0x7f5fac113190 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.207+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1afba0 con 0x7f5fac113190 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.208+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac10db60 con 0x7f5fac113190 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.209+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 17) ==== 103285+0+0 (secure 0 0 0) 0x7f5f9c003fb0 con 0x7f5fac113190 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.209+0000 7f5fa2ffd640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5f9807b0f0 0x7f5f9807d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.209+0000 7f5fb1bfe640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5f9807b0f0 0x7f5f9807d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.209+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f5f9c09e840 con 0x7f5fac113190 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.210+0000 7f5fb1bfe640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5f9807b0f0 0x7f5f9807d5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8005fd0 tx=0x7f5fa800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:00.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.214+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f9c069960 con 0x7f5fac113190 2024-09-21T11:54:00.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.393+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fac1afe60 con 0x7f5fac113190 2024-09-21T11:54:00.395 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:00.396 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:00.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.395+0000 7f5fa2ffd640 1 -- 172.21.15.107:0/2003431568 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+22821 (secure 0 0 0) 0x7f5f9c06e810 con 0x7f5fac113190 2024-09-21T11:54:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:54:00.397 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":63,"fsid":"1795c04c-780f-11ef-baf6-efdc52797490","created":"2024-09-21T11:47:57.300122+0000","modified":"2024-09-21T11:53:47.211709+0000","last_up_change":"2024-09-21T11:53:27.207544+0000","last_in_change":"2024-09-21T11:53:04.078828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:51:05.622296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-21T11:53:34.884714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-21T11:53:35.804433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-21T11:53:37.455475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-21T11:53:39.397850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-21T11:53:41.458869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e4feb96-c3c7-4f2d-9ba3-1db963213296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6803","nonce":2032034052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6805","nonce":2032034052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6809","nonce":2032034052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2032034052},{"type":"v1","addr":"172.21.15.107:6807","nonce":2032034052}]},"public_addr":"172.21.15.107:6803/2032034052","cluster_addr":"172.21.15.107:6805/2032034052","heartbeat_back_addr":"172.21.15.107:6809/2032034052","heartbeat_front_addr":"172.21.15.107:6807/2032034052","state":["exists","up"]},{"osd":1,"uuid":"a464ccec-f1a1-48ae-b233-89519142c774","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6811","nonce":3491646662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6813","nonce":3491646662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6817","nonce":3491646662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3491646662},{"type":"v1","addr":"172.21.15.107:6815","nonce":3491646662}]},"public_addr":"172.21.15.107:6811/3491646662","cluster_addr":"172.21.15.107:6813/3491646662","heartbeat_back_addr":"172.21.15.107:6817/3491646662","heartbeat_front_addr":"172.21.15.107:6815/3491646662","state":["exists","up"]},{"osd":2,"uuid":"74584ac0-8833-4c1e-9f98-2bef0780286e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6819","nonce":216243970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6821","nonce":216243970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6825","nonce":216243970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":216243970},{"type":"v1","addr":"172.21.15.107:6823","nonce":216243970}]},"public_addr":"172.21.15.107:6819/216243970","cluster_addr":"172.21.15.107:6821/216243970","heartbeat_back_addr":"172.21.15.107:6825/216243970","heartbeat_front_addr":"172.21.15.107:6823/216243970","state":["exists","up"]},{"osd":3,"uuid":"661db9ba-cabf-4159-b051-379aa09acc00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6827","nonce":1625163749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6829","nonce":1625163749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6833","nonce":1625163749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1625163749},{"type":"v1","addr":"172.21.15.107:6831","nonce":1625163749}]},"public_addr":"172.21.15.107:6827/1625163749","cluster_addr":"172.21.15.107:6829/1625163749","heartbeat_back_addr":"172.21.15.107:6833/1625163749","heartbeat_front_addr":"172.21.15.107:6831/1625163749","state":["exists","up"]},{"osd":4,"uuid":"6c71418f-c007-4428-b338-3ebfc82f779d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6801","nonce":3995503105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6803","nonce":3995503105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6807","nonce":3995503105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3995503105},{"type":"v1","addr":"172.21.15.193:6805","nonce":3995503105}]},"public_addr":"172.21.15.193:6801/3995503105","cluster_addr":"172.21.15.193:6803/3995503105","heartbeat_back_addr":"172.21.15.193:6807/3995503105","heartbeat_front_addr":"172.21.15.193:6805/3995503105","state":["exists","up"]},{"osd":5,"uuid":"f2576976-ac0c-45e0-8353-95220342328d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6809","nonce":102264086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6811","nonce":102264086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6815","nonce":102264086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":102264086},{"type":"v1","addr":"172.21.15.193:6813","nonce":102264086}]},"public_addr":"172.21.15.193:6809/102264086","cluster_addr":"172.21.15.193:6811/102264086","heartbeat_back_addr":"172.21.15.193:6815/102264086","heartbeat_front_addr":"172.21.15.193:6813/102264086","state":["exists","up"]},{"osd":6,"uuid":"14c736d5-64dc-41f2-9cc8-6b8adaba0ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6817","nonce":1896699774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6819","nonce":1896699774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6823","nonce":1896699774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1896699774},{"type":"v1","addr":"172.21.15.193:6821","nonce":1896699774}]},"public_addr":"172.21.15.193:6817/1896699774","cluster_addr":"172.21.15.193:6819/1896699774","heartbeat_back_addr":"172.21.15.193:6823/1896699774","heartbeat_front_addr":"172.21.15.193:6821/1896699774","state":["exists","up"]},{"osd":7,"uuid":"fb16cb03-fe65-4189-818c-29a99c0db388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6825","nonce":338565813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6827","nonce":338565813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6831","nonce":338565813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":338565813},{"type":"v1","addr":"172.21.15.193:6829","nonce":338565813}]},"public_addr":"172.21.15.193:6825/338565813","cluster_addr":"172.21.15.193:6827/338565813","heartbeat_back_addr":"172.21.15.193:6831/338565813","heartbeat_front_addr":"172.21.15.193:6829/338565813","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:50:02.844467+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:50:32.631011+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:02.098307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:31.483117+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:51:58.119895+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:52:27.691211+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:52:56.093529+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:53:25.370332+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3723772931":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6800/1807990149":"2024-09-22T11:48:45.320004+0000","172.21.15.107:0/1627664470":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6800/1943709907":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/3703659764":"2024-09-22T11:48:45.320004+0000","172.21.15.107:0/545710521":"2024-09-22T11:48:19.247117+0000","172.21.15.107:6801/1807990149":"2024-09-22T11:48:45.320004+0000","172.21.15.107:6801/1943709907":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/2867069233":"2024-09-22T11:48:19.247117+0000","172.21.15.107:0/1351066913":"2024-09-22T11:48:19.247117+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T11:54:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5f9807b0f0 msgr2=0x7f5f9807d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5f9807b0f0 0x7f5f9807d5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8005fd0 tx=0x7f5fa800b4a0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 msgr2=0x7f5fac1a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:00.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 0x7f5fac1a3680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e9f0 tx=0x7f5f9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 shutdown_connections 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f5f9807b0f0 0x7f5f9807d5b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fac113b10 0x7f5fac1a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5fac113190 0x7f5fac1a3680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 --2- 172.21.15.107:0/2003431568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5fac10d780 0x7f5fac1a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 >> 172.21.15.107:0/2003431568 conn(0x7f5fac06dc80 msgr2=0x7f5fac10ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 shutdown_connections 2024-09-21T11:54:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:00.397+0000 7f5fb3e62640 1 -- 172.21.15.107:0/2003431568 wait complete. 2024-09-21T11:54:00.493 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:00.800 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 podman[61638]: 2024-09-21 11:54:00.495371107 +0000 UTC m=+0.116810746 container create 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:01.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T11:54:01.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T11:54:01.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T11:54:01.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T11:54:01.061 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T11:54:01.061 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T11:54:01.062 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T11:54:01.062 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 podman[61638]: 2024-09-21 11:54:00.801908051 +0000 UTC m=+0.423347690 container init 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 podman[61638]: 2024-09-21 11:54:00.806509753 +0000 UTC m=+0.427949392 container start 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.826Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.826Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.826Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi193 (none))" 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.826Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.826Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.828Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-21T11:54:01.301 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.828Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.830Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.830Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.832Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.832Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.975µs 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.832Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.832Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.832Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.784µs wal_replay_duration=161.089µs wbl_replay_duration=104ns total_replay_duration=202.033µs 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.833Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.833Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.833Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.849Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.840312ms db_storage=701ns remote_storage=1.088µs web_handler=275ns query_engine=474ns scrape=424.555µs scrape_sd=85.21µs notify=919ns notify_sd=780ns rules=15.035461ms tracing=5.833µs 2024-09-21T11:54:01.302 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.849Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-21T11:54:01.303 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:54:00.849Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-21T11:54:01.303 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 bash[61638]: 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 2024-09-21T11:54:01.303 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:54:00 smithi193 systemd[1]: Started Ceph prometheus.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:54:01.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[30293]: pgmap v210: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1.4 KiB/s wr, 48 op/s 2024-09-21T11:54:01.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2003431568' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:54:01.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.334 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-21T11:54:01.602 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.605 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.606 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.606 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.606 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.609 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.609 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.611 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:01.737 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[41097]: pgmap v210: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1.4 KiB/s wr, 48 op/s 2024-09-21T11:54:01.737 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2003431568' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:54:01.737 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.737 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.737 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.737 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:01 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-21T11:54:01.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:01 smithi193 ceph-mon[33333]: pgmap v210: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1.4 KiB/s wr, 48 op/s 2024-09-21T11:54:01.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2003431568' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:54:01.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' 2024-09-21T11:54:01.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:01 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-21T11:54:02.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:02 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: ignoring --setuser ceph since I am not root 2024-09-21T11:54:02.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:02 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: ignoring --setgroup ceph since I am not root 2024-09-21T11:54:02.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:02 smithi193 ceph-mgr[35028]: -- 172.21.15.193:0/4072703312 <== mon.2 v2:172.21.15.107:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ded5ed0e0 con 0x557ded5cb000 2024-09-21T11:54:02.367 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ignoring --setuser ceph since I am not root 2024-09-21T11:54:02.367 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ignoring --setgroup ceph since I am not root 2024-09-21T11:54:02.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 -- 172.21.15.107:0/2167641535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93581087d0 msgr2=0x7f9358108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:02.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 --2- 172.21.15.107:0/2167641535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93581087d0 0x7f9358108c10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9340009860 tx=0x7f934002a340 comp rx=0 tx=0).stop 2024-09-21T11:54:02.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 -- 172.21.15.107:0/2167641535 shutdown_connections 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 --2- 172.21.15.107:0/2167641535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9358109150 0x7f935810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 --2- 172.21.15.107:0/2167641535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93581087d0 0x7f9358108c10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 --2- 172.21.15.107:0/2167641535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9358102dc0 0x7f93581031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 -- 172.21.15.107:0/2167641535 >> 172.21.15.107:0/2167641535 conn(0x7f93580fe810 msgr2=0x7f9358100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 -- 172.21.15.107:0/2167641535 shutdown_connections 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.533+0000 7f935ea1c640 1 -- 172.21.15.107:0/2167641535 wait complete. 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 Processor -- start 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 -- start start 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9358102dc0 0x7f93581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93581087d0 0x7f9358198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9358109150 0x7f935819e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9358112270 con 0x7f93581087d0 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93581120f0 con 0x7f9358102dc0 2024-09-21T11:54:02.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935ea1c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f93581123f0 con 0x7f9358109150 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f93577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93581087d0 0x7f9358198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f935cfb9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9358109150 0x7f935819e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f93577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93581087d0 0x7f9358198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57728/0 (socket says 172.21.15.107:57728) 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f93577fe640 1 -- 172.21.15.107:0/4130425942 learned_addr learned my addr 172.21.15.107:0/4130425942 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.534+0000 7f9357fff640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9358102dc0 0x7f93581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93577fe640 1 -- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9358109150 msgr2=0x7f935819e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93577fe640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9358109150 0x7f935819e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93577fe640 1 -- 172.21.15.107:0/4130425942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9358102dc0 msgr2=0x7f93581987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93577fe640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9358102dc0 0x7f93581987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93577fe640 1 -- 172.21.15.107:0/4130425942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935819e890 con 0x7f93581087d0 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f9357fff640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9358102dc0 0x7f93581987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f935cfb9640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9358109150 0x7f935819e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:02.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93577fe640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93581087d0 0x7f9358198d20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93400057c0 tx=0x7f9340033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340037040 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935819eb20 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9358075380 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9340040070 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.535+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340004490 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.536+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9358063860 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.537+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 103299+0+0 (secure 0 0 0) 0x7f9340030920 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f93557fa640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f934407b140 0x7f934407d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f9357fff640 1 -- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f934407b140 msgr2=0x7f934407d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2130999972 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f93400be2b0 con 0x7f93581087d0 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f9357fff640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f934407b140 0x7f934407d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f93557fa640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] conn(0x7f9344084c90 0x7f93440870f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 --> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93440877c0 con 0x7f9344084c90 2024-09-21T11:54:02.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f93400c5050 con 0x7f93581087d0 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f935cfb9640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] conn(0x7f9344084c90 0x7f93440870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.538+0000 7f935cfb9640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] conn(0x7f9344084c90 0x7f93440870f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.539+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== osd.2 v2:172.21.15.107:6818/216243970 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93440877c0 con 0x7f9344084c90 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 --> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9358063c60 con 0x7f9344084c90 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f93557fa640 1 -- 172.21.15.107:0/4130425942 <== osd.2 v2:172.21.15.107:6818/216243970 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9358063c60 con 0x7f9344084c90 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] conn(0x7f9344084c90 msgr2=0x7f93440870f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6818/216243970,v1:172.21.15.107:6819/216243970] conn(0x7f9344084c90 0x7f93440870f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f934407b140 msgr2=0x7f934407d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f934407b140 0x7f934407d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:02.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93581087d0 msgr2=0x7f9358198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 --2- 172.21.15.107:0/4130425942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93581087d0 0x7f9358198d20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93400057c0 tx=0x7f9340033d00 comp rx=0 tx=0).stop 2024-09-21T11:54:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935cfb9640 1 -- 172.21.15.107:0/4130425942 reap_dead start 2024-09-21T11:54:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 shutdown_connections 2024-09-21T11:54:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 >> 172.21.15.107:0/4130425942 conn(0x7f93580fe810 msgr2=0x7f9358100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 shutdown_connections 2024-09-21T11:54:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:02.553+0000 7f935ea1c640 1 -- 172.21.15.107:0/4130425942 wait complete. 2024-09-21T11:54:03.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:02 smithi107 ceph-mon[41097]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T11:54:03.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:02 smithi107 ceph-mon[41097]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-21T11:54:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:02 smithi107 ceph-mon[30293]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T11:54:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:02 smithi107 ceph-mon[30293]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-21T11:54:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:02 smithi193 ceph-mon[33333]: from='mgr.14152 172.21.15.107:0/1948709593' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T11:54:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:02 smithi193 ceph-mon[33333]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 -- 172.21.15.107:0/1194608501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c101c60 msgr2=0x7fc48c102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 --2- 172.21.15.107:0/1194608501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c101c60 0x7fc48c102040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009900 tx=0x7fc47c02a410 comp rx=0 tx=0).stop 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 -- 172.21.15.107:0/1194608501 shutdown_connections 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 --2- 172.21.15.107:0/1194608501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c107ff0 0x7fc48c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 --2- 172.21.15.107:0/1194608501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c107670 0x7fc48c107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 --2- 172.21.15.107:0/1194608501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c101c60 0x7fc48c102040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 -- 172.21.15.107:0/1194608501 >> 172.21.15.107:0/1194608501 conn(0x7fc48c0fd6d0 msgr2=0x7fc48c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:03.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 -- 172.21.15.107:0/1194608501 shutdown_connections 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 -- 172.21.15.107:0/1194608501 wait complete. 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 Processor -- start 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.417+0000 7fc49234a640 1 -- start start 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc49234a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c101c60 0x7fc48c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc49234a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c107670 0x7fc48c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc49234a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c107ff0 0x7fc48c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc49234a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c111180 con 0x7fc48c107670 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc49234a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c111000 con 0x7fc48c107ff0 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc49234a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc48c111300 con 0x7fc48c101c60 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c101c60 0x7fc48c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c107670 0x7fc48c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c101c60 0x7fc48c1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:55784/0 (socket says 172.21.15.107:55784) 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 -- 172.21.15.107:0/12441284 learned_addr learned my addr 172.21.15.107:0/12441284 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:03.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc4908e7640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c107ff0 0x7fc48c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 -- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c107ff0 msgr2=0x7fc48c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c107ff0 0x7fc48c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 -- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c107670 msgr2=0x7fc48c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c107670 0x7fc48c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.418+0000 7fc48bfff640 1 -- 172.21.15.107:0/12441284 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c1a49b0 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.419+0000 7fc48bfff640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c101c60 0x7fc48c1995d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc47c005940 tx=0x7fc47c032b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.419+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c03f030 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.419+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c1a4c40 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.419+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc47c003a40 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.419+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c035070 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.419+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c1a5170 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.420+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc48c063860 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 18) ==== 103299+0+0 (secure 0 0 0) 0x7fc47c042030 con 0x7fc48c101c60 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4897fa640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc47407b140 0x7fc47407d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc48b7fe640 1 -- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc47407b140 msgr2=0x7fc47407d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2130999972 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7fc47c0bf210 con 0x7fc48c101c60 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc48b7fe640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc47407b140 0x7fc47407d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4897fa640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] conn(0x7fc474084c10 0x7fc474087070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 --> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc47c03be50 con 0x7fc474084c10 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fc47c0bf6f0 con 0x7fc48c101c60 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.421+0000 7fc4908e7640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] conn(0x7fc474084c10 0x7fc474087070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.422+0000 7fc4908e7640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] conn(0x7fc474084c10 0x7fc474087070 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:03.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.423+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== osd.4 v2:172.21.15.193:6800/3995503105 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc47c03be50 con 0x7fc474084c10 2024-09-21T11:54:03.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.436+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 --> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc48c063c60 con 0x7fc474084c10 2024-09-21T11:54:03.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc4897fa640 1 -- 172.21.15.107:0/12441284 <== osd.4 v2:172.21.15.193:6800/3995503105 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc48c063c60 con 0x7fc474084c10 2024-09-21T11:54:03.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] conn(0x7fc474084c10 msgr2=0x7fc474087070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.193:6800/3995503105,v1:172.21.15.193:6801/3995503105] conn(0x7fc474084c10 0x7fc474087070 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc47407b140 msgr2=0x7fc47407d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7fc47407b140 0x7fc47407d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c101c60 msgr2=0x7fc48c1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 --2- 172.21.15.107:0/12441284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c101c60 0x7fc48c1995d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc47c005940 tx=0x7fc47c032b70 comp rx=0 tx=0).stop 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc4908e7640 1 -- 172.21.15.107:0/12441284 reap_dead start 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 shutdown_connections 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.437+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 >> 172.21.15.107:0/12441284 conn(0x7fc48c0fd6d0 msgr2=0x7fc48c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.438+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 shutdown_connections 2024-09-21T11:54:03.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:03.438+0000 7fc49234a640 1 -- 172.21.15.107:0/12441284 wait complete. 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 -- 172.21.15.107:0/1665615451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 msgr2=0x7f68ec1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 --2- 172.21.15.107:0/1665615451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec1031a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009860 tx=0x7f68dc02a340 comp rx=0 tx=0).stop 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 -- 172.21.15.107:0/1665615451 shutdown_connections 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 --2- 172.21.15.107:0/1665615451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec109150 0x7f68ec10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 --2- 172.21.15.107:0/1665615451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68ec1087d0 0x7f68ec108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 --2- 172.21.15.107:0/1665615451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec1031a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 -- 172.21.15.107:0/1665615451 >> 172.21.15.107:0/1665615451 conn(0x7f68ec0fe850 msgr2=0x7f68ec100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 -- 172.21.15.107:0/1665615451 shutdown_connections 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 -- 172.21.15.107:0/1665615451 wait complete. 2024-09-21T11:54:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.093+0000 7f68f1234640 1 Processor -- start 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 -- start start 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec1087d0 0x7f68ec19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68ec109150 0x7f68ec1a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec1122e0 con 0x7f68ec1087d0 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec112160 con 0x7f68ec109150 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68f1234640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f68ec112460 con 0x7f68ec102dc0 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ea59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec1087d0 0x7f68ec19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:55796/0 (socket says 172.21.15.107:55796) 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 -- 172.21.15.107:0/3710538152 learned_addr learned my addr 172.21.15.107:0/3710538152 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68eb59e640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68ec109150 0x7f68ec1a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 -- 172.21.15.107:0/3710538152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68ec109150 msgr2=0x7f68ec1a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68ec109150 0x7f68ec1a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 -- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec1087d0 msgr2=0x7f68ec19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec1087d0 0x7f68ec19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ead9d640 1 -- 172.21.15.107:0/3710538152 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec1a0ab0 con 0x7f68ec102dc0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.094+0000 7f68ea59c640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec1087d0 0x7f68ec19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.095+0000 7f68ead9d640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec19aa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68dc035040 tx=0x7f68dc033d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.095+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc03f020 con 0x7f68ec102dc0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.095+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f68dc0306b0 con 0x7f68ec102dc0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.096+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc037040 con 0x7f68ec102dc0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.096+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec1a0d40 con 0x7f68ec102dc0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.096+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec1a7460 con 0x7f68ec102dc0 2024-09-21T11:54:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.097+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68ec1009a0 con 0x7f68ec102dc0 2024-09-21T11:54:04.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.097+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 18) ==== 103299+0+0 (secure 0 0 0) 0x7f68dc042020 con 0x7f68ec102dc0 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68d3fff640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f68d407b140 0x7f68d407d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68ea59c640 1 -- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f68d407b140 msgr2=0x7f68d407d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2130999972 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f68dc0be180 con 0x7f68ec102dc0 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68ea59c640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f68d407b140 0x7f68d407d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68d3fff640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] conn(0x7f68d4084c90 0x7f68d40870f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 --> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68dc03aec0 con 0x7f68d4084c90 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.098+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f68dc0c5050 con 0x7f68ec102dc0 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.099+0000 7f68eb59e640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] conn(0x7f68d4084c90 0x7f68d40870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.099+0000 7f68eb59e640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] conn(0x7f68d4084c90 0x7f68d40870f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:04.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.100+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== osd.1 v2:172.21.15.107:6810/3491646662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68dc03aec0 con 0x7f68d4084c90 2024-09-21T11:54:04.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.127+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 --> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68ec104070 con 0x7f68d4084c90 2024-09-21T11:54:04.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68d3fff640 1 -- 172.21.15.107:0/3710538152 <== osd.1 v2:172.21.15.107:6810/3491646662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f68ec104070 con 0x7f68d4084c90 2024-09-21T11:54:04.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] conn(0x7f68d4084c90 msgr2=0x7f68d40870f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6810/3491646662,v1:172.21.15.107:6811/3491646662] conn(0x7f68d4084c90 0x7f68d40870f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f68d407b140 msgr2=0x7f68d407d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:54:04.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f68d407b140 0x7f68d407d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 msgr2=0x7f68ec19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 --2- 172.21.15.107:0/3710538152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68ec102dc0 0x7f68ec19aa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68dc035040 tx=0x7f68dc033d00 comp rx=0 tx=0).stop 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68eb59e640 1 -- 172.21.15.107:0/3710538152 reap_dead start 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 shutdown_connections 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 >> 172.21.15.107:0/3710538152 conn(0x7f68ec0fe850 msgr2=0x7f68ec100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 shutdown_connections 2024-09-21T11:54:04.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.128+0000 7f68f1234640 1 -- 172.21.15.107:0/3710538152 wait complete. 2024-09-21T11:54:04.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 -- 172.21.15.107:0/3570092864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3960103c70 msgr2=0x7f3960104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 --2- 172.21.15.107:0/3570092864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3960103c70 0x7f3960104050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3950009860 tx=0x7f395002a340 comp rx=0 tx=0).stop 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 -- 172.21.15.107:0/3570092864 shutdown_connections 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 --2- 172.21.15.107:0/3570092864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f396010d0d0 0x7f396010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 --2- 172.21.15.107:0/3570092864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3960104590 0x7f396010ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 --2- 172.21.15.107:0/3570092864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3960103c70 0x7f3960104050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 -- 172.21.15.107:0/3570092864 >> 172.21.15.107:0/3570092864 conn(0x7f39600fd6d0 msgr2=0x7f39600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 -- 172.21.15.107:0/3570092864 shutdown_connections 2024-09-21T11:54:04.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.564+0000 7f3964b44640 1 -- 172.21.15.107:0/3570092864 wait complete. 2024-09-21T11:54:04.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 Processor -- start 2024-09-21T11:54:04.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 -- start start 2024-09-21T11:54:04.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3960103c70 0x7f3960198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3960104590 0x7f3960198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f396010d0d0 0x7f396019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3960112210 con 0x7f3960104590 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3960112090 con 0x7f3960103c70 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f3964b44640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3960112390 con 0x7f396010d0d0 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f395dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3960104590 0x7f3960198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f395ed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f396010d0d0 0x7f396019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f395ed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f396010d0d0 0x7f396019e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:47584/0 (socket says 172.21.15.107:47584) 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.565+0000 7f395ed9d640 1 -- 172.21.15.107:0/3750553687 learned_addr learned my addr 172.21.15.107:0/3750553687 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395e59c640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3960103c70 0x7f3960198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395dd9b640 1 -- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f396010d0d0 msgr2=0x7f396019e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395dd9b640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f396010d0d0 0x7f396019e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395dd9b640 1 -- 172.21.15.107:0/3750553687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3960103c70 msgr2=0x7f3960198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395dd9b640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3960103c70 0x7f3960198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395dd9b640 1 -- 172.21.15.107:0/3750553687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396019e8e0 con 0x7f3960104590 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395ed9d640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f396010d0d0 0x7f396019e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f395dd9b640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3960104590 0x7f3960198d70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f395400e790 tx=0x7f395400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395400eea0 con 0x7f3960104590 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3954004480 con 0x7f3960104590 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954003cc0 con 0x7f3960104590 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396019ebd0 con 0x7f3960104590 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.566+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3960101800 con 0x7f3960104590 2024-09-21T11:54:04.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.567+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3960106410 con 0x7f3960104590 2024-09-21T11:54:04.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.568+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 103299+0+0 (secure 0 0 0) 0x7f3954003e60 con 0x7f3960104590 2024-09-21T11:54:04.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f394f7fe640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f394807b140 0x7f394807d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f395e59c640 1 -- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f394807b140 msgr2=0x7f394807d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2130999972 2024-09-21T11:54:04.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f395409e8c0 con 0x7f3960104590 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f395e59c640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f394807b140 0x7f394807d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f394f7fe640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] conn(0x7f3948084c90 0x7f39480870f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 --> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39480877c0 con 0x7f3948084c90 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f39540a32c0 con 0x7f3960104590 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f395ed9d640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] conn(0x7f3948084c90 0x7f39480870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.569+0000 7f395ed9d640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] conn(0x7f3948084c90 0x7f39480870f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:04.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.570+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== osd.0 v2:172.21.15.107:6802/2032034052 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39480877c0 con 0x7f3948084c90 2024-09-21T11:54:04.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.598+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 --> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f396006b180 con 0x7f3948084c90 2024-09-21T11:54:04.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.598+0000 7f394f7fe640 1 -- 172.21.15.107:0/3750553687 <== osd.0 v2:172.21.15.107:6802/2032034052 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f396006b180 con 0x7f3948084c90 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] conn(0x7f3948084c90 msgr2=0x7f39480870f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6802/2032034052,v1:172.21.15.107:6803/2032034052] conn(0x7f3948084c90 0x7f39480870f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f394807b140 msgr2=0x7f394807d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f394807b140 0x7f394807d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3960104590 msgr2=0x7f3960198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 --2- 172.21.15.107:0/3750553687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3960104590 0x7f3960198d70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f395400e790 tx=0x7f395400ec50 comp rx=0 tx=0).stop 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f395ed9d640 1 -- 172.21.15.107:0/3750553687 reap_dead start 2024-09-21T11:54:04.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 shutdown_connections 2024-09-21T11:54:04.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 >> 172.21.15.107:0/3750553687 conn(0x7f39600fd6d0 msgr2=0x7f3960104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:04.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 shutdown_connections 2024-09-21T11:54:04.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:04.599+0000 7f3964b44640 1 -- 172.21.15.107:0/3750553687 wait complete. 2024-09-21T11:54:05.194 INFO:teuthology.orchestra.run.smithi107.stdout:55834574893 2024-09-21T11:54:05.194 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T11:54:05.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/3594814380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d441087d0 msgr2=0x7f3d44108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:05.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/3594814380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d441087d0 0x7f3d44108c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009900 tx=0x7f3d3802a410 comp rx=0 tx=0).stop 2024-09-21T11:54:05.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/3594814380 shutdown_connections 2024-09-21T11:54:05.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/3594814380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d44109150 0x7f3d4410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/3594814380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d441087d0 0x7f3d44108c10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/3594814380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d44102dc0 0x7f3d441031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/3594814380 >> 172.21.15.107:0/3594814380 conn(0x7f3d440fe850 msgr2=0x7f3d44100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/3594814380 shutdown_connections 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.391+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/3594814380 wait complete. 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 Processor -- start 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 -- start start 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d44102dc0 0x7f3d4410ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d441087d0 0x7f3d4410f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109150 0x7f3d441a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d441122f0 con 0x7f3d44109150 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d44112170 con 0x7f3d44102dc0 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.392+0000 7f3d4a7b5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3d44112470 con 0x7f3d441087d0 2024-09-21T11:54:05.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109150 0x7f3d441a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d437fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d441087d0 0x7f3d4410f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109150 0x7f3d441a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40534/0 (socket says 172.21.15.107:40534) 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 -- 172.21.15.107:0/1115789700 learned_addr learned my addr 172.21.15.107:0/1115789700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d43fff640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d44102dc0 0x7f3d4410ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 -- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d441087d0 msgr2=0x7f3d4410f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d441087d0 0x7f3d4410f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 -- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d44102dc0 msgr2=0x7f3d4410ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d44102dc0 0x7f3d4410ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 -- 172.21.15.107:0/1115789700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d441a1c40 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d48d52640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109150 0x7f3d441a14c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00eaa0 tx=0x7f3d3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00cdb0 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3d3c005020 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d441a1f30 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.393+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0116d0 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.394+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d441ab8a0 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.395+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d44063860 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.396+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 103299+0+0 (secure 0 0 0) 0x7f3d3c004a40 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.396+0000 7f3d417fa640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3d2c07b140 0x7f3d2c07d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.396+0000 7f3d43fff640 1 -- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3d2c07b140 msgr2=0x7f3d2c07d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2130999972 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.396+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 7148+0+0 (secure 0 0 0) 0x7f3d3c09e8a0 con 0x7f3d44109150 2024-09-21T11:54:05.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.396+0000 7f3d43fff640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3d2c07b140 0x7f3d2c07d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.397+0000 7f3d417fa640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] conn(0x7f3d2c084c90 0x7f3d2c0870f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.397+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 --> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d2c0877c0 con 0x7f3d2c084c90 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.397+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f3d3c0a32c0 con 0x7f3d44109150 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.397+0000 7f3d437fe640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] conn(0x7f3d2c084c90 0x7f3d2c0870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.397+0000 7f3d437fe640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] conn(0x7f3d2c084c90 0x7f3d2c0870f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.398+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== osd.6 v2:172.21.15.193:6816/1896699774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d2c0877c0 con 0x7f3d2c084c90 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.413+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 --> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d44063c60 con 0x7f3d2c084c90 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d417fa640 1 -- 172.21.15.107:0/1115789700 <== osd.6 v2:172.21.15.193:6816/1896699774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3d44063c60 con 0x7f3d2c084c90 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] conn(0x7f3d2c084c90 msgr2=0x7f3d2c0870f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.193:6816/1896699774,v1:172.21.15.193:6817/1896699774] conn(0x7f3d2c084c90 0x7f3d2c0870f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3d2c07b140 msgr2=0x7f3d2c07d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:6800/2130999972,v1:172.21.15.107:6801/2130999972] conn(0x7f3d2c07b140 0x7f3d2c07d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109150 msgr2=0x7f3d441a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:05.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 --2- 172.21.15.107:0/1115789700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d44109150 0x7f3d441a14c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00eaa0 tx=0x7f3d3c00ef60 comp rx=0 tx=0).stop 2024-09-21T11:54:05.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d48d52640 1 -- 172.21.15.107:0/1115789700 reap_dead start 2024-09-21T11:54:05.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 shutdown_connections 2024-09-21T11:54:05.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.414+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 >> 172.21.15.107:0/1115789700 conn(0x7f3d440fe850 msgr2=0x7f3d44104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:05.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.415+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 shutdown_connections 2024-09-21T11:54:05.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:05.415+0000 7f3d4a7b5640 1 -- 172.21.15.107:0/1115789700 wait complete. 2024-09-21T11:54:06.293 INFO:teuthology.orchestra.run.smithi107.stdout:184683593743 2024-09-21T11:54:06.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T11:54:06.944 INFO:teuthology.orchestra.run.smithi107.stdout:34359738418 2024-09-21T11:54:06.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T11:54:07.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[30293]: Standby manager daemon x restarted 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[30293]: Standby manager daemon x started 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[30293]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[41097]: Standby manager daemon x restarted 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[41097]: Standby manager daemon x started 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:54:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:54:07.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:54:07.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:07 smithi107 ceph-mon[41097]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:54:07.300 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: [21/Sep/2024:11:54:06] ENGINE Bus STARTING 2024-09-21T11:54:07.300 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: CherryPy Checker: 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: The Application mounted at '' has an empty config. 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: [21/Sep/2024:11:54:07] ENGINE Serving on http://:::9283 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x[35005]: [21/Sep/2024:11:54:07] ENGINE Bus STARTED 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-mon[33333]: Standby manager daemon x restarted 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-mon[33333]: Standby manager daemon x started 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:54:07.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:07 smithi193 ceph-mon[33333]: from='mgr.? 172.21.15.193:0/4154219290' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:54:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:133143986203 2024-09-21T11:54:07.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.556+0000 7f710e5a0640 1 -- 172.21.15.107:0/526355686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081018c0 msgr2=0x7f7108101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.556+0000 7f710e5a0640 1 --2- 172.21.15.107:0/526355686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081018c0 0x7f7108101ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009900 tx=0x7f70f802a440 comp rx=0 tx=0).stop 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.557+0000 7f710e5a0640 1 -- 172.21.15.107:0/526355686 shutdown_connections 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.557+0000 7f710e5a0640 1 --2- 172.21.15.107:0/526355686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f7108107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.557+0000 7f710e5a0640 1 --2- 172.21.15.107:0/526355686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71081021e0 0x7f71081066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.557+0000 7f710e5a0640 1 --2- 172.21.15.107:0/526355686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081018c0 0x7f7108101ca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.557+0000 7f710e5a0640 1 -- 172.21.15.107:0/526355686 >> 172.21.15.107:0/526355686 conn(0x7f71080fd570 msgr2=0x7f71080ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.558+0000 7f710e5a0640 1 -- 172.21.15.107:0/526355686 shutdown_connections 2024-09-21T11:54:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.558+0000 7f710e5a0640 1 -- 172.21.15.107:0/526355686 wait complete. 2024-09-21T11:54:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 Processor -- start 2024-09-21T11:54:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 -- start start 2024-09-21T11:54:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71081018c0 0x7f710819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081021e0 0x7f710819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f71081a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7108118760 con 0x7f71081021e0 2024-09-21T11:54:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71081185e0 con 0x7f7108106d00 2024-09-21T11:54:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.559+0000 7f710e5a0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f71081188e0 con 0x7f71081018c0 2024-09-21T11:54:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f71081a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710cd9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081021e0 0x7f710819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f71081a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:48366/0 (socket says 172.21.15.107:48366) 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 -- 172.21.15.107:0/280768948 learned_addr learned my addr 172.21.15.107:0/280768948 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 -- 172.21.15.107:0/280768948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71081018c0 msgr2=0x7f710819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71081018c0 0x7f710819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 -- 172.21.15.107:0/280768948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081021e0 msgr2=0x7f710819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081021e0 0x7f710819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.560+0000 7f710dd9f640 1 -- 172.21.15.107:0/280768948 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081a4d50 con 0x7f7108106d00 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.561+0000 7f710dd9f640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f71081a4580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f710000b990 tx=0x7f710000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.561+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100019070 con 0x7f7108106d00 2024-09-21T11:54:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.561+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7100014440 con 0x7f7108106d00 2024-09-21T11:54:07.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.561+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000075d0 con 0x7f7108106d00 2024-09-21T11:54:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.561+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71081a5040 con 0x7f7108106d00 2024-09-21T11:54:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.561+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081ab780 con 0x7f7108106d00 2024-09-21T11:54:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.562+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7108105960 con 0x7f7108106d00 2024-09-21T11:54:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.563+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 102915+0+0 (secure 0 0 0) 0x7f710001c070 con 0x7f7108106d00 2024-09-21T11:54:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.564+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f710009d7c0 con 0x7f7108106d00 2024-09-21T11:54:07.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.564+0000 7f70fe7fc640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] conn(0x7f70f4082080 0x7f70f40844c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.564+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 --> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7100021e50 con 0x7f70f4082080 2024-09-21T11:54:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.564+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f71000a22a0 con 0x7f7108106d00 2024-09-21T11:54:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.564+0000 7f710d59e640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] conn(0x7f70f4082080 0x7f70f40844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.565+0000 7f710d59e640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] conn(0x7f70f4082080 0x7f70f40844c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:07.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.565+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== osd.7 v2:172.21.15.193:6824/338565813 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7100021e50 con 0x7f70f4082080 2024-09-21T11:54:07.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.588+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 --> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f710806b9a0 con 0x7f70f4082080 2024-09-21T11:54:07.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.589+0000 7f70fe7fc640 1 -- 172.21.15.107:0/280768948 <== osd.7 v2:172.21.15.193:6824/338565813 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f710806b9a0 con 0x7f70f4082080 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] conn(0x7f70f4082080 msgr2=0x7f70f40844c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] conn(0x7f70f4082080 0x7f70f40844c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 msgr2=0x7f71081a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f71081a4580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f710000b990 tx=0x7f710000be50 comp rx=0 tx=0).stop 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 shutdown_connections 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:6824/338565813,v1:172.21.15.193:6825/338565813] conn(0x7f70f4082080 0x7f70f40844c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7108106d00 0x7f71081a4580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71081021e0 0x7f710819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 --2- 172.21.15.107:0/280768948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71081018c0 0x7f710819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 >> 172.21.15.107:0/280768948 conn(0x7f71080fd570 msgr2=0x7f71080ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:07.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 shutdown_connections 2024-09-21T11:54:07.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.590+0000 7f710e5a0640 1 -- 172.21.15.107:0/280768948 wait complete. 2024-09-21T11:54:07.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 -- 172.21.15.107:0/4152159177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c101c90 msgr2=0x7f176c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 --2- 172.21.15.107:0/4152159177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c101c90 0x7f176c102070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1768009860 tx=0x7f176802a340 comp rx=0 tx=0).stop 2024-09-21T11:54:07.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 -- 172.21.15.107:0/4152159177 shutdown_connections 2024-09-21T11:54:07.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 --2- 172.21.15.107:0/4152159177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f176c108020 0x7f176c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 --2- 172.21.15.107:0/4152159177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c1076a0 0x7f176c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 --2- 172.21.15.107:0/4152159177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c101c90 0x7f176c102070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 -- 172.21.15.107:0/4152159177 >> 172.21.15.107:0/4152159177 conn(0x7f176c0fd720 msgr2=0x7f176c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.773+0000 7f17739c7640 1 -- 172.21.15.107:0/4152159177 shutdown_connections 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.774+0000 7f17739c7640 1 -- 172.21.15.107:0/4152159177 wait complete. 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.774+0000 7f17739c7640 1 Processor -- start 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 -- start start 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 0x7f176c110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f176c1076a0 0x7f176c10f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c108020 0x7f176c10f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f176c112290 con 0x7f176c101c90 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 0x7f176c110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:07.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f176c112110 con 0x7f176c108020 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 0x7f176c110c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40644/0 (socket says 172.21.15.107:40644) 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 -- 172.21.15.107:0/1449304797 learned_addr learned my addr 172.21.15.107:0/1449304797 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f176c112410 con 0x7f176c1076a0 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 -- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f176c1076a0 msgr2=0x7f176c10f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f176c1076a0 0x7f176c10f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 -- 172.21.15.107:0/1449304797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c108020 msgr2=0x7f176c10f7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c108020 0x7f176c10f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.775+0000 7f1771763640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c1ab0f0 con 0x7f176c101c90 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.776+0000 7f1771763640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 0x7f176c110c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f176802a860 tx=0x7f1768033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.776+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176803f020 con 0x7f176c101c90 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.776+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f1768004490 con 0x7f176c101c90 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.776+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768037040 con 0x7f176c101c90 2024-09-21T11:54:07.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.776+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c1ab2c0 con 0x7f176c101c90 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.776+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1ab660 con 0x7f176c101c90 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.778+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f176c063860 con 0x7f176c101c90 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.778+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 102915+0+0 (secure 0 0 0) 0x7f1768042020 con 0x7f176c101c90 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.779+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f17680bdfb0 con 0x7f176c101c90 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.779+0000 7f175e7fc640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] conn(0x7f1758082050 0x7f1758084490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.779+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1768043e70 con 0x7f1758082050 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.779+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f17680c5050 con 0x7f176c101c90 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.779+0000 7f1770f62640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] conn(0x7f1758082050 0x7f1758084490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.780+0000 7f1770f62640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] conn(0x7f1758082050 0x7f1758084490 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:07.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.781+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== osd.3 v2:172.21.15.107:6826/1625163749 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1768043e70 con 0x7f1758082050 2024-09-21T11:54:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.800+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 --> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f176c063c60 con 0x7f1758082050 2024-09-21T11:54:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.800+0000 7f175e7fc640 1 -- 172.21.15.107:0/1449304797 <== osd.3 v2:172.21.15.107:6826/1625163749 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f176c063c60 con 0x7f1758082050 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.800+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] conn(0x7f1758082050 msgr2=0x7f1758084490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.800+0000 7f17739c7640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] conn(0x7f1758082050 0x7f1758084490 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 msgr2=0x7f176c110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 0x7f176c110c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f176802a860 tx=0x7f1768033d10 comp rx=0 tx=0).stop 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 shutdown_connections 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:6826/1625163749,v1:172.21.15.107:6827/1625163749] conn(0x7f1758082050 0x7f1758084490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f176c108020 0x7f176c10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f176c1076a0 0x7f176c10f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 --2- 172.21.15.107:0/1449304797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f176c101c90 0x7f176c110c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 >> 172.21.15.107:0/1449304797 conn(0x7f176c0fd720 msgr2=0x7f176c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 shutdown_connections 2024-09-21T11:54:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:07.801+0000 7f17739c7640 1 -- 172.21.15.107:0/1449304797 wait complete. 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: Active manager daemon y restarted 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: Activating manager daemon y 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: osdmap e64: 8 total, 8 up, 8 in 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: mgrmap e20: y(active, starting, since 0.00924759s), standbys: x 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:54:08.120 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: Manager daemon y is now available 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:08.121 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: Active manager daemon y restarted 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: Activating manager daemon y 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: osdmap e64: 8 total, 8 up, 8 in 2024-09-21T11:54:08.122 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: mgrmap e20: y(active, starting, since 0.00924759s), standbys: x 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:54:08.123 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: Manager daemon y is now available 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:54:08.124 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:54:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.116+0000 7effd9431640 1 -- 172.21.15.107:0/1832472308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 msgr2=0x7effd4102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.116+0000 7effd9431640 1 --2- 172.21.15.107:0/1832472308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd4102cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a410 comp rx=0 tx=0).stop 2024-09-21T11:54:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.116+0000 7effd9431640 1 -- 172.21.15.107:0/1832472308 shutdown_connections 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.117+0000 7effd9431640 1 --2- 172.21.15.107:0/1832472308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd41080e0 0x7effd410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.117+0000 7effd9431640 1 --2- 172.21.15.107:0/1832472308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd4107740 0x7effd4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.117+0000 7effd9431640 1 --2- 172.21.15.107:0/1832472308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd4102cc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.117+0000 7effd9431640 1 -- 172.21.15.107:0/1832472308 >> 172.21.15.107:0/1832472308 conn(0x7effd4077ee0 msgr2=0x7effd4079640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.118+0000 7effd9431640 1 -- 172.21.15.107:0/1832472308 shutdown_connections 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.118+0000 7effd9431640 1 -- 172.21.15.107:0/1832472308 wait complete. 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.119+0000 7effd9431640 1 Processor -- start 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.119+0000 7effd9431640 1 -- start start 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.120+0000 7effd9431640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd41974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.120+0000 7effd9431640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd4107740 0x7effd4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.120+0000 7effd2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd41974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.120+0000 7effd2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd41974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:48442/0 (socket says 172.21.15.107:48442) 2024-09-21T11:54:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.120+0000 7effd2ffd640 1 -- 172.21.15.107:0/881711494 learned_addr learned my addr 172.21.15.107:0/881711494 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd27fc640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd4107740 0x7effd4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd41080e0 0x7effd419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7effd4111180 con 0x7effd41080e0 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7effd4111000 con 0x7effd41028e0 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7effd4111300 con 0x7effd4107740 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd37fe640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd41080e0 0x7effd419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd2ffd640 1 -- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd4107740 msgr2=0x7effd4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd2ffd640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd4107740 0x7effd4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd2ffd640 1 -- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd41080e0 msgr2=0x7effd419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd2ffd640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd41080e0 0x7effd419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd2ffd640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd419d5a0 con 0x7effd41028e0 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.121+0000 7effd2ffd640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd41974f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7effc4009850 tx=0x7effc4005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:08.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.122+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035030 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.122+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7effc4041070 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.122+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4030a60 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.122+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd419d830 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.122+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd4101570 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.123+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effd4100b50 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.124+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 102915+0+0 (secure 0 0 0) 0x7effc4030c00 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.125+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7effc40c2ba0 con 0x7effd41028e0 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.125+0000 7effbbfff640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] conn(0x7effbc081f80 0x7effbc0843c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.125+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effc402fec0 con 0x7effbc081f80 2024-09-21T11:54:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.125+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7effc40c8240 con 0x7effd41028e0 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.125+0000 7effd27fc640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] conn(0x7effbc081f80 0x7effbc0843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.125+0000 7effd27fc640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] conn(0x7effbc081f80 0x7effbc0843c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.126+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== osd.5 v2:172.21.15.193:6808/102264086 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effc402fec0 con 0x7effbc081f80 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.152+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 --> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7effd406ad80 con 0x7effbc081f80 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.152+0000 7effbbfff640 1 -- 172.21.15.107:0/881711494 <== osd.5 v2:172.21.15.193:6808/102264086 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7effd406ad80 con 0x7effbc081f80 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] conn(0x7effbc081f80 msgr2=0x7effbc0843c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] conn(0x7effbc081f80 0x7effbc0843c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 msgr2=0x7effd41974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd41974f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7effc4009850 tx=0x7effc4005e10 comp rx=0 tx=0).stop 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 shutdown_connections 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effd41080e0 0x7effd419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:6808/102264086,v1:172.21.15.193:6809/102264086] conn(0x7effbc081f80 0x7effbc0843c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effd4107740 0x7effd4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 --2- 172.21.15.107:0/881711494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7effd41028e0 0x7effd41974f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:08.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 >> 172.21.15.107:0/881711494 conn(0x7effd4077ee0 msgr2=0x7effd41001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:08.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 shutdown_connections 2024-09-21T11:54:08.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:08.153+0000 7effd9431640 1 -- 172.21.15.107:0/881711494 wait complete. 2024-09-21T11:54:08.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: Active manager daemon y restarted 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: Activating manager daemon y 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: osdmap e64: 8 total, 8 up, 8 in 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: mgrmap e20: y(active, starting, since 0.00924759s), standbys: x 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:54:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: Manager daemon y is now available 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:54:08.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:54:08.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:54:08.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:54:08.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:54:08.422 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:08] ENGINE Bus STARTING 2024-09-21T11:54:08.422 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: CherryPy Checker: 2024-09-21T11:54:08.422 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: The Application mounted at '' has an empty config. 2024-09-21T11:54:08.422 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:54:08.422 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:08] ENGINE Serving on http://:::9283 2024-09-21T11:54:08.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:54:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:54:08.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:08 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:08] ENGINE Bus STARTED 2024-09-21T11:54:08.891 INFO:teuthology.orchestra.run.smithi107.stdout:158913789972 2024-09-21T11:54:08.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T11:54:09.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[30293]: mgrmap e21: y(active, since 1.01304s), standbys: x 2024-09-21T11:54:09.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[30293]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:09.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:54:09.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:09.462 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[41097]: mgrmap e21: y(active, since 1.01304s), standbys: x 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[41097]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:54:09.198+0000 7f4439d52640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:11:54:09] ENGINE Error in HTTPServer.serve 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: Traceback (most recent call last): 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._connections.run(self.expiration_interval) 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._run(expiration_interval) 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:54:09.463 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: s = self.context.wrap_socket( 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: return self.sslsocket_class._create( 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self.do_handshake() 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: self._sslobj.do_handshake() 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:54:09.464 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T11:54:09.548 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:09 smithi193 ceph-mon[33333]: mgrmap e21: y(active, since 1.01304s), standbys: x 2024-09-21T11:54:09.549 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:09 smithi193 ceph-mon[33333]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:09.549 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:09 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:54:09.549 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:09.798 INFO:teuthology.orchestra.run.smithi107.stdout:111669149728 2024-09-21T11:54:09.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T11:54:10.295 INFO:teuthology.orchestra.run.smithi107.stdout:77309411367 2024-09-21T11:54:10.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T11:54:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: [21/Sep/2024:11:54:09] ENGINE Bus STARTING 2024-09-21T11:54:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: [21/Sep/2024:11:54:09] ENGINE Serving on https://172.21.15.107:7150 2024-09-21T11:54:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: [21/Sep/2024:11:54:09] ENGINE Error in HTTPServer.serve 2024-09-21T11:54:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._run(expiration_interval) 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: s = self.context.wrap_socket( 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.sslsocket_class._create( 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:54:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout: self.do_handshake() 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._sslobj.do_handshake() 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: [21/Sep/2024:11:54:09] ENGINE Serving on http://172.21.15.107:8765 2024-09-21T11:54:10.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:10 smithi193 ceph-mon[33333]: [21/Sep/2024:11:54:09] ENGINE Bus STARTED 2024-09-21T11:54:10.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: [21/Sep/2024:11:54:09] ENGINE Bus STARTING 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: [21/Sep/2024:11:54:09] ENGINE Serving on https://172.21.15.107:7150 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: [21/Sep/2024:11:54:09] ENGINE Error in HTTPServer.serve 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:54:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-21T11:54:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: [21/Sep/2024:11:54:09] ENGINE Serving on http://172.21.15.107:8765 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[30293]: [21/Sep/2024:11:54:09] ENGINE Bus STARTED 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: [21/Sep/2024:11:54:09] ENGINE Bus STARTING 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: [21/Sep/2024:11:54:09] ENGINE Serving on https://172.21.15.107:7150 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: [21/Sep/2024:11:54:09] ENGINE Error in HTTPServer.serve 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: Traceback (most recent call last): 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: self._run(expiration_interval) 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:54:10.756 INFO:journalctl@ceph.mon.c.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: s = self.context.wrap_socket( 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: return self.sslsocket_class._create( 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: self.do_handshake() 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: self._sslobj.do_handshake() 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: [21/Sep/2024:11:54:09] ENGINE Serving on http://172.21.15.107:8765 2024-09-21T11:54:10.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:10 smithi107 ceph-mon[41097]: [21/Sep/2024:11:54:09] ENGINE Bus STARTED 2024-09-21T11:54:10.932 INFO:teuthology.orchestra.run.smithi107.stdout:214748364808 2024-09-21T11:54:10.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T11:54:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:10.958 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:11.249 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 -- 172.21.15.107:0/2612945702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 msgr2=0x7f534c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 --2- 172.21.15.107:0/2612945702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c108bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5340009900 tx=0x7f534002a410 comp rx=0 tx=0).stop 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 -- 172.21.15.107:0/2612945702 shutdown_connections 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 --2- 172.21.15.107:0/2612945702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 --2- 172.21.15.107:0/2612945702 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c108bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 --2- 172.21.15.107:0/2612945702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c102da0 0x7f534c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 -- 172.21.15.107:0/2612945702 >> 172.21.15.107:0/2612945702 conn(0x7f534c0fe810 msgr2=0x7f534c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 -- 172.21.15.107:0/2612945702 shutdown_connections 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.440+0000 7f5352a5e640 1 -- 172.21.15.107:0/2612945702 wait complete. 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 Processor -- start 2024-09-21T11:54:11.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 -- start start 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c102da0 0x7f534c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1122c0 con 0x7f534c109130 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f534c112140 con 0x7f534c102da0 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5352a5e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f534c112440 con 0x7f534c1087b0 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f534b7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5350ffb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f534bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c102da0 0x7f534c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5350ffb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c19e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40674/0 (socket says 172.21.15.107:40674) 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f534b7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:47752/0 (socket says 172.21.15.107:47752) 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.441+0000 7f5350ffb640 1 -- 172.21.15.107:0/1034623390 learned_addr learned my addr 172.21.15.107:0/1034623390 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5350ffb640 1 -- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 msgr2=0x7f534c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5350ffb640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5350ffb640 1 -- 172.21.15.107:0/1034623390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c102da0 msgr2=0x7f534c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5350ffb640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c102da0 0x7f534c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5350ffb640 1 -- 172.21.15.107:0/1034623390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c19e880 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5350ffb640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c19e0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f534400efd0 tx=0x7f534400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344003b70 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c19eb70 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f5344003d10 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344010730 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.442+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1a5230 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.443+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534c104050 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.444+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 103236+0+0 (secure 0 0 0) 0x7f5344004020 con 0x7f534c109130 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.444+0000 7f53497fa640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f533407b0c0 0x7f533407d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:11.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.444+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f534409ea90 con 0x7f534c109130 2024-09-21T11:54:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.444+0000 7f534bfff640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f533407b0c0 0x7f533407d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.444+0000 7f534bfff640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f533407b0c0 0x7f533407d580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f533c00bbe0 tx=0x7f533c00b390 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.449+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5344069c80 con 0x7f534c109130 2024-09-21T11:54:11.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.635+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f534c199900 con 0x7f534c109130 2024-09-21T11:54:11.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.635+0000 7f53497fa640 1 -- 172.21.15.107:0/1034623390 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f534406eb30 con 0x7f534c109130 2024-09-21T11:54:11.638 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f533407b0c0 msgr2=0x7f533407d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f533407b0c0 0x7f533407d580 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f533c00bbe0 tx=0x7f533c00b390 comp rx=0 tx=0).stop 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 msgr2=0x7f534c19e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c19e0b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f534400efd0 tx=0x7f534400c6a0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 shutdown_connections 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f533407b0c0 0x7f533407d580 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c109130 0x7f534c19e0b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.637+0000 7f5352a5e640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f534c1087b0 0x7f534c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.638+0000 7f5352a5e640 1 --2- 172.21.15.107:0/1034623390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f534c102da0 0x7f534c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.638+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 >> 172.21.15.107:0/1034623390 conn(0x7f534c0fe810 msgr2=0x7f534c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.638+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 shutdown_connections 2024-09-21T11:54:11.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:11.638+0000 7f5352a5e640 1 -- 172.21.15.107:0/1034623390 wait complete. 2024-09-21T11:54:11.988 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.988 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.988 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.988 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.988 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.988 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:11.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:11 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:11 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:12.291 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574894 for osd.1 2024-09-21T11:54:12.292 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:12.564 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:12.565 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:12.565 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:12.578 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:54:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:12.578 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi193 to 3760M 2024-09-21T11:54:12.578 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[30293]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:12.579 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1034623390' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T11:54:12.579 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[30293]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-21T11:54:13.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi193 to 3760M 2024-09-21T11:54:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[41097]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1034623390' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T11:54:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:12 smithi107 ceph-mon[41097]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-21T11:54:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:12 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi193 to 3760M 2024-09-21T11:54:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:12 smithi193 ceph-mon[33333]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1034623390' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T11:54:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:12 smithi193 ceph-mon[33333]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-21T11:54:13.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 -- 172.21.15.107:0/2533363728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa760101880 msgr2=0x7fa760101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:13.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 --2- 172.21.15.107:0/2533363728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa760101880 0x7fa760101c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa754009860 tx=0x7fa75402a340 comp rx=0 tx=0).stop 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 -- 172.21.15.107:0/2533363728 shutdown_connections 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 --2- 172.21.15.107:0/2533363728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 0x7fa760111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 --2- 172.21.15.107:0/2533363728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa7601021a0 0x7fa76010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 --2- 172.21.15.107:0/2533363728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa760101880 0x7fa760101c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 -- 172.21.15.107:0/2533363728 >> 172.21.15.107:0/2533363728 conn(0x7fa7600fd6d0 msgr2=0x7fa7600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 -- 172.21.15.107:0/2533363728 shutdown_connections 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.580+0000 7fa766de7640 1 -- 172.21.15.107:0/2533363728 wait complete. 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 Processor -- start 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 -- start start 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 0x7fa76019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7601021a0 0x7fa76019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 0x7fa7601a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa760114460 con 0x7fa7601021a0 2024-09-21T11:54:13.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601142e0 con 0x7fa76010f430 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa766de7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fa7601145e0 con 0x7fa760101880 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa764b83640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 0x7fa76019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa75ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7601021a0 0x7fa76019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa764b83640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 0x7fa76019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:47764/0 (socket says 172.21.15.107:47764) 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.581+0000 7fa764b83640 1 -- 172.21.15.107:0/3052518890 learned_addr learned my addr 172.21.15.107:0/3052518890 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa765384640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 0x7fa7601a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75ffff640 1 -- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 msgr2=0x7fa76019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75ffff640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 0x7fa76019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75ffff640 1 -- 172.21.15.107:0/3052518890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 msgr2=0x7fa7601a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75ffff640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 0x7fa7601a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75ffff640 1 -- 172.21.15.107:0/3052518890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7601a0aa0 con 0x7fa7601021a0 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa764b83640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 0x7fa76019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa765384640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 0x7fa7601a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:54:13.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75ffff640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7601021a0 0x7fa76019af30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005fd0 tx=0x7fa74c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00cd00 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fa74c004470 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7601a0d90 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c003db0 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.582+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a7450 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.583+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa760101c60 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.584+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7fa74c004000 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.584+0000 7fa75dffb640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fa75007b150 0x7fa75007d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.584+0000 7fa764b83640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fa75007b150 0x7fa75007d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.584+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7fa74c09ea40 con 0x7fa7601021a0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.585+0000 7fa764b83640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fa75007b150 0x7fa75007d610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa7540057c0 tx=0x7fa7540056d0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:13.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.588+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa74c069b50 con 0x7fa7601021a0 2024-09-21T11:54:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.773+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa76010a5c0 con 0x7fa7601021a0 2024-09-21T11:54:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.773+0000 7fa75dffb640 1 -- 172.21.15.107:0/3052518890 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa74c06ea00 con 0x7fa7601021a0 2024-09-21T11:54:13.775 INFO:teuthology.orchestra.run.smithi107.stdout:158913789973 2024-09-21T11:54:13.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fa75007b150 msgr2=0x7fa75007d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:13.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fa75007b150 0x7fa75007d610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa7540057c0 tx=0x7fa7540056d0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7601021a0 msgr2=0x7fa76019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:13.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7601021a0 0x7fa76019af30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005fd0 tx=0x7fa74c00ee50 comp rx=0 tx=0).stop 2024-09-21T11:54:13.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 shutdown_connections 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fa75007b150 0x7fa75007d610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa76010f430 0x7fa7601a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7601021a0 0x7fa76019af30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 --2- 172.21.15.107:0/3052518890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa760101880 0x7fa76019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 >> 172.21.15.107:0/3052518890 conn(0x7fa7600fd6d0 msgr2=0x7fa7600ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 shutdown_connections 2024-09-21T11:54:13.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:13.775+0000 7fa766de7640 1 -- 172.21.15.107:0/3052518890 wait complete. 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 -- 172.21.15.107:0/695463435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1045b0 msgr2=0x7f263c10caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 --2- 172.21.15.107:0/695463435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1045b0 0x7f263c10caa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2624009860 tx=0x7f262402a340 comp rx=0 tx=0).stop 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 -- 172.21.15.107:0/695463435 shutdown_connections 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 --2- 172.21.15.107:0/695463435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f263c10d0f0 0x7f263c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 --2- 172.21.15.107:0/695463435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1045b0 0x7f263c10caa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 --2- 172.21.15.107:0/695463435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c103c90 0x7f263c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 -- 172.21.15.107:0/695463435 >> 172.21.15.107:0/695463435 conn(0x7f263c0fd6f0 msgr2=0x7f263c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 -- 172.21.15.107:0/695463435 shutdown_connections 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.226+0000 7f2642d14640 1 -- 172.21.15.107:0/695463435 wait complete. 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 Processor -- start 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 -- start start 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 0x7f263c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 0x7f263c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f263c10d0f0 0x7f263c1a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f263c112230 con 0x7f263c1045b0 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f263c1120b0 con 0x7f263c10d0f0 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2642d14640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f263c1123b0 con 0x7f263c103c90 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f263bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 0x7f263c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2640ab0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 0x7f263c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2640ab0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 0x7f263c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44496/0 (socket says 172.21.15.107:44496) 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f2640ab0640 1 -- 172.21.15.107:0/2980333664 learned_addr learned my addr 172.21.15.107:0/2980333664 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:14.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.227+0000 7f263bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 0x7f263c19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43072/0 (socket says 172.21.15.107:43072) 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2640ab0640 1 -- 172.21.15.107:0/2980333664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f263c10d0f0 msgr2=0x7f263c1a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2640ab0640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f263c10d0f0 0x7f263c1a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2640ab0640 1 -- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 msgr2=0x7f263c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2640ab0640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 0x7f263c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2640ab0640 1 -- 172.21.15.107:0/2980333664 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c1a0ab0 con 0x7f263c103c90 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f263bfff640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 0x7f263c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2640ab0640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 0x7f263c19aa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c420 tx=0x7f262c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c003970 con 0x7f263c103c90 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c1a0da0 con 0x7f263c103c90 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f262c004020 con 0x7f263c103c90 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c00b870 con 0x7f263c103c90 2024-09-21T11:54:14.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.228+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c1a7460 con 0x7f263c103c90 2024-09-21T11:54:14.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.229+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c104070 con 0x7f263c103c90 2024-09-21T11:54:14.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.230+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f262c01d020 con 0x7f263c103c90 2024-09-21T11:54:14.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.230+0000 7f2639ffb640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f262807b150 0x7f262807d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.230+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f262c09e760 con 0x7f263c103c90 2024-09-21T11:54:14.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.230+0000 7f263bfff640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f262807b150 0x7f262807d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.231+0000 7f263bfff640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f262807b150 0x7f262807d610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2624009830 tx=0x7f26240058d0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:14.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.235+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f262c069870 con 0x7f263c103c90 2024-09-21T11:54:14.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.413+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f263c1a7690 con 0x7f263c103c90 2024-09-21T11:54:14.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.413+0000 7f2639ffb640 1 -- 172.21.15.107:0/2980333664 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f262c06e720 con 0x7f263c103c90 2024-09-21T11:54:14.415 INFO:teuthology.orchestra.run.smithi107.stdout:111669149729 2024-09-21T11:54:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f262807b150 msgr2=0x7f262807d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f262807b150 0x7f262807d610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2624009830 tx=0x7f26240058d0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 msgr2=0x7f263c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 0x7f263c19aa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c420 tx=0x7f262c009f90 comp rx=0 tx=0).stop 2024-09-21T11:54:14.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 shutdown_connections 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f262807b150 0x7f262807d610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f263c10d0f0 0x7f263c1a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f263c1045b0 0x7f263c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 --2- 172.21.15.107:0/2980333664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c103c90 0x7f263c19aa00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 >> 172.21.15.107:0/2980333664 conn(0x7f263c0fd6f0 msgr2=0x7f263c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.415+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 shutdown_connections 2024-09-21T11:54:14.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.416+0000 7f2642d14640 1 -- 172.21.15.107:0/2980333664 wait complete. 2024-09-21T11:54:14.677 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:14 smithi107 ceph-mon[41097]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:14.677 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3052518890' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T11:54:14.677 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2980333664' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T11:54:14.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:14 smithi107 ceph-mon[30293]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:14.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3052518890' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T11:54:14.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2980333664' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T11:54:14.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 -- 172.21.15.107:0/325838882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 msgr2=0x7f9e7c10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 --2- 172.21.15.107:0/325838882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c10f910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009ca0 tx=0x7f9e7402a8c0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 -- 172.21.15.107:0/325838882 shutdown_connections 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 --2- 172.21.15.107:0/325838882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c10f910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 --2- 172.21.15.107:0/325838882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c108790 0x7f9e7c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 --2- 172.21.15.107:0/325838882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c102d80 0x7f9e7c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 -- 172.21.15.107:0/325838882 >> 172.21.15.107:0/325838882 conn(0x7f9e7c0fe7d0 msgr2=0x7f9e7c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 -- 172.21.15.107:0/325838882 shutdown_connections 2024-09-21T11:54:14.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.518+0000 7f9e8118a640 1 -- 172.21.15.107:0/325838882 wait complete. 2024-09-21T11:54:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 Processor -- start 2024-09-21T11:54:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 -- start start 2024-09-21T11:54:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c102d80 0x7f9e7c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 0x7f9e7c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c112260 con 0x7f9e7c109110 2024-09-21T11:54:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c1120e0 con 0x7f9e7c102d80 2024-09-21T11:54:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e8118a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c1123e0 con 0x7f9e7c108790 2024-09-21T11:54:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e7a59c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 0x7f9e7c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e7b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.795 INFO:teuthology.orchestra.run.smithi107.stdout:77309411368 2024-09-21T11:54:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e7b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c19e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43094/0 (socket says 172.21.15.107:43094) 2024-09-21T11:54:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e7a59c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 0x7f9e7c198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44512/0 (socket says 172.21.15.107:44512) 2024-09-21T11:54:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.519+0000 7f9e7b59e640 1 -- 172.21.15.107:0/1324635590 learned_addr learned my addr 172.21.15.107:0/1324635590 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7ad9d640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c102d80 0x7f9e7c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7b59e640 1 -- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 msgr2=0x7f9e7c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7b59e640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 0x7f9e7c198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7b59e640 1 -- 172.21.15.107:0/1324635590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c102d80 msgr2=0x7f9e7c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7b59e640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c102d80 0x7f9e7c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7b59e640 1 -- 172.21.15.107:0/1324635590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c19e880 con 0x7f9e7c109110 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7a59c640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 0x7f9e7c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7ad9d640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c102d80 0x7f9e7c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e7b59e640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c19e0b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e740097b0 tx=0x7f9e74003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74003e10 con 0x7f9e7c109110 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c19eb10 con 0x7f9e7c109110 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9e74004490 con 0x7f9e7c109110 2024-09-21T11:54:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74030ac0 con 0x7f9e7c109110 2024-09-21T11:54:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.520+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c1a5230 con 0x7f9e7c109110 2024-09-21T11:54:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.521+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c103160 con 0x7f9e7c109110 2024-09-21T11:54:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.522+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f9e74030c60 con 0x7f9e7c109110 2024-09-21T11:54:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.522+0000 7f9e63fff640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e6407b150 0x7f9e6407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.522+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f9e740bf990 con 0x7f9e7c109110 2024-09-21T11:54:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.522+0000 7f9e7ad9d640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e6407b150 0x7f9e6407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.523+0000 7f9e7ad9d640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e6407b150 0x7f9e6407d610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009b50 tx=0x7f9e6c005f90 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.527+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e7408abb0 con 0x7f9e7c109110 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.732+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9e7c10aef0 con 0x7f9e7c109110 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.732+0000 7f9e63fff640 1 -- 172.21.15.107:0/1324635590 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e7408fa60 con 0x7f9e7c109110 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e6407b150 msgr2=0x7f9e6407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e6407b150 0x7f9e6407d610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009b50 tx=0x7f9e6c005f90 comp rx=0 tx=0).stop 2024-09-21T11:54:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 msgr2=0x7f9e7c19e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c19e0b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e740097b0 tx=0x7f9e74003c20 comp rx=0 tx=0).stop 2024-09-21T11:54:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 shutdown_connections 2024-09-21T11:54:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e6407b150 0x7f9e6407d610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e7c109110 0x7f9e7c19e0b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e7c108790 0x7f9e7c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 --2- 172.21.15.107:0/1324635590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e7c102d80 0x7f9e7c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 >> 172.21.15.107:0/1324635590 conn(0x7f9e7c0fe7d0 msgr2=0x7f9e7c106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 shutdown_connections 2024-09-21T11:54:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:14.734+0000 7f9e8118a640 1 -- 172.21.15.107:0/1324635590 wait complete. 2024-09-21T11:54:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:14 smithi193 ceph-mon[33333]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:54:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3052518890' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T11:54:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2980333664' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T11:54:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2216156777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 msgr2=0x7f7c1410f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2216156777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1410f910 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2216156777 shutdown_connections 2024-09-21T11:54:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2216156777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1410f910 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2216156777 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2216156777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c14102d80 0x7f7c14103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2216156777 >> 172.21.15.107:0/2216156777 conn(0x7f7c140fe7d0 msgr2=0x7f7c14100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2216156777 shutdown_connections 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.198+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2216156777 wait complete. 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 Processor -- start 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 -- start start 2024-09-21T11:54:15.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c14102d80 0x7f7c14198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:15.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:15.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:15.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141122a0 con 0x7f7c14109110 2024-09-21T11:54:15.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14112120 con 0x7f7c14102d80 2024-09-21T11:54:15.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.199+0000 7f7c1bae3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7c14112420 con 0x7f7c14108790 2024-09-21T11:54:15.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1a080640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44528/0 (socket says 172.21.15.107:44528) 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 -- 172.21.15.107:0/2642594913 learned_addr learned my addr 172.21.15.107:0/2642594913 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1987f640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c14102d80 0x7f7c14198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 -- 172.21.15.107:0/2642594913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c14102d80 msgr2=0x7f7c14198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c14102d80 0x7f7c14198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 -- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 msgr2=0x7f7c1419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 -- 172.21.15.107:0/2642594913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1419e7e0 con 0x7f7c14108790 2024-09-21T11:54:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1a080640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1907e640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14198c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400ea60 tx=0x7f7c0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400cde0 con 0x7f7c14108790 2024-09-21T11:54:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7c04004470 con 0x7f7c14108790 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1419ead0 con 0x7f7c14108790 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04010430 con 0x7f7c14108790 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.200+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141a5190 con 0x7f7c14108790 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.201+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14103160 con 0x7f7c14108790 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.202+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f7c04003f60 con 0x7f7c14108790 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.202+0000 7f7c0affd640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c0007b150 0x7f7c0007d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.202+0000 7f7c1987f640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c0007b150 0x7f7c0007d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:15.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.202+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f7c0409eac0 con 0x7f7c14108790 2024-09-21T11:54:15.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.203+0000 7f7c1987f640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c0007b150 0x7f7c0007d610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c100058c0 tx=0x7f7c1000a2d0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:15.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.207+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c04069bd0 con 0x7f7c14108790 2024-09-21T11:54:15.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.392+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7c1410aef0 con 0x7f7c14108790 2024-09-21T11:54:15.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.393+0000 7f7c0affd640 1 -- 172.21.15.107:0/2642594913 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c0406ea80 con 0x7f7c14108790 2024-09-21T11:54:15.395 INFO:teuthology.orchestra.run.smithi107.stdout:184683593744 2024-09-21T11:54:15.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c0007b150 msgr2=0x7f7c0007d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:15.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c0007b150 0x7f7c0007d610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7c100058c0 tx=0x7f7c1000a2d0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 msgr2=0x7f7c14198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:15.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14198c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400ea60 tx=0x7f7c0400ef20 comp rx=0 tx=0).stop 2024-09-21T11:54:15.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 shutdown_connections 2024-09-21T11:54:15.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c0007b150 0x7f7c0007d610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c14109110 0x7f7c1419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c14108790 0x7f7c14198c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 --2- 172.21.15.107:0/2642594913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c14102d80 0x7f7c14198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:15.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 >> 172.21.15.107:0/2642594913 conn(0x7f7c140fe7d0 msgr2=0x7f7c14100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:15.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 shutdown_connections 2024-09-21T11:54:15.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:15.396+0000 7f7c1bae3640 1 -- 172.21.15.107:0/2642594913 wait complete. 2024-09-21T11:54:15.727 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1324635590' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T11:54:15.727 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2642594913' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T11:54:15.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1324635590' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T11:54:15.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2642594913' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T11:54:16.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789972 got 158913789973 for osd.5 2024-09-21T11:54:16.031 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1324635590' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T11:54:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2642594913' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 -- 172.21.15.107:0/3195690271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b901076b0 msgr2=0x7f3b90107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 --2- 172.21.15.107:0/3195690271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b901076b0 0x7f3b90107af0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009900 tx=0x7f3b8002a410 comp rx=0 tx=0).stop 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 -- 172.21.15.107:0/3195690271 shutdown_connections 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 --2- 172.21.15.107:0/3195690271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b90108030 0x7f3b9010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 --2- 172.21.15.107:0/3195690271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b901076b0 0x7f3b90107af0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 --2- 172.21.15.107:0/3195690271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b90102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 -- 172.21.15.107:0/3195690271 >> 172.21.15.107:0/3195690271 conn(0x7f3b900fd6f0 msgr2=0x7f3b900ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.327+0000 7f3b982e3640 1 -- 172.21.15.107:0/3195690271 shutdown_connections 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 -- 172.21.15.107:0/3195690271 wait complete. 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 Processor -- start 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 -- start start 2024-09-21T11:54:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b900fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b901076b0 0x7f3b900fe130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b90108030 0x7f3b900fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90111150 con 0x7f3b90108030 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b90110fd0 con 0x7f3b901076b0 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b982e3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f3b901112d0 con 0x7f3b90101ca0 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b9607f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b900fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b96880640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b90108030 0x7f3b900fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b9607f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b900fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44548/0 (socket says 172.21.15.107:44548) 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.328+0000 7f3b9607f640 1 -- 172.21.15.107:0/1213866846 learned_addr learned my addr 172.21.15.107:0/1213866846 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9587e640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b901076b0 0x7f3b900fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9607f640 1 -- 172.21.15.107:0/1213866846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b901076b0 msgr2=0x7f3b900fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9607f640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b901076b0 0x7f3b900fe130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9607f640 1 -- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b90108030 msgr2=0x7f3b900fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:16.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9607f640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b90108030 0x7f3b900fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9607f640 1 -- 172.21.15.107:0/1213866846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b901a6bb0 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b96880640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b90108030 0x7f3b900fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b9607f640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b900fdbf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ea60 tx=0x7f3b8c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00cde0 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f3b8c004470 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b901a6ea0 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c010430 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.329+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b901a7300 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.330+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b90102080 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.331+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f3b8c003f60 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.331+0000 7f3b877fe640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f3b7c07b150 0x7f3b7c07d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.331+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f3b8c09eae0 con 0x7f3b90101ca0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.331+0000 7f3b9587e640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f3b7c07b150 0x7f3b7c07d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:16.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.332+0000 7f3b9587e640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f3b7c07b150 0x7f3b7c07d610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3b80006010 tx=0x7f3b80005780 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.335+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b8c069bf0 con 0x7f3b90101ca0 2024-09-21T11:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.523+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3b9010ae70 con 0x7f3b90101ca0 2024-09-21T11:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.523+0000 7f3b877fe640 1 -- 172.21.15.107:0/1213866846 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b9010ae70 con 0x7f3b90101ca0 2024-09-21T11:54:16.526 INFO:teuthology.orchestra.run.smithi107.stdout:133143986205 2024-09-21T11:54:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f3b7c07b150 msgr2=0x7f3b7c07d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f3b7c07b150 0x7f3b7c07d610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3b80006010 tx=0x7f3b80005780 comp rx=0 tx=0).stop 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 msgr2=0x7f3b900fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b900fdbf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00ea60 tx=0x7f3b8c00ef20 comp rx=0 tx=0).stop 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 shutdown_connections 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f3b7c07b150 0x7f3b7c07d610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b90108030 0x7f3b900fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b901076b0 0x7f3b900fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 --2- 172.21.15.107:0/1213866846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b90101ca0 0x7f3b900fdbf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 >> 172.21.15.107:0/1213866846 conn(0x7f3b900fd6f0 msgr2=0x7f3b9006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 shutdown_connections 2024-09-21T11:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:16.526+0000 7f3b982e3640 1 -- 172.21.15.107:0/1213866846 wait complete. 2024-09-21T11:54:17.002 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:16 smithi107 ceph-mon[41097]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T11:54:17.002 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1213866846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T11:54:17.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:16 smithi107 ceph-mon[30293]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T11:54:17.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1213866846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T11:54:17.028 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149728 got 111669149729 for osd.3 2024-09-21T11:54:17.028 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:16 smithi193 ceph-mon[33333]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T11:54:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1213866846' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T11:54:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 -- 172.21.15.107:0/1290069366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928810d780 msgr2=0x7f928810db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 --2- 172.21.15.107:0/1290069366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928810d780 0x7f928810db60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9284009900 tx=0x7f928402a410 comp rx=0 tx=0).stop 2024-09-21T11:54:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 -- 172.21.15.107:0/1290069366 shutdown_connections 2024-09-21T11:54:17.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 --2- 172.21.15.107:0/1290069366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9288113b10 0x7f928811a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 --2- 172.21.15.107:0/1290069366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113190 0x7f92881135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 --2- 172.21.15.107:0/1290069366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f928810d780 0x7f928810db60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 -- 172.21.15.107:0/1290069366 >> 172.21.15.107:0/1290069366 conn(0x7f928806dc80 msgr2=0x7f928806e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 -- 172.21.15.107:0/1290069366 shutdown_connections 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 -- 172.21.15.107:0/1290069366 wait complete. 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 Processor -- start 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 -- start start 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.199+0000 7f9290662640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f928810d780 0x7f92881a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288113190 0x7f92881a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113b10 0x7f92881a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f928811cc50 con 0x7f9288113190 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f928811cad0 con 0x7f9288113b10 2024-09-21T11:54:17.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f928811cdd0 con 0x7f928810d780 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928e3fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f928810d780 0x7f92881a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928e3fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f928810d780 0x7f92881a31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44566/0 (socket says 172.21.15.107:44566) 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928e3fe640 1 -- 172.21.15.107:0/2421294388 learned_addr learned my addr 172.21.15.107:0/2421294388 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288113190 0x7f92881a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928ebff640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113b10 0x7f92881a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 -- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f928810d780 msgr2=0x7f92881a31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f928810d780 0x7f92881a31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 -- 172.21.15.107:0/2421294388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113b10 msgr2=0x7f92881a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113b10 0x7f92881a8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 -- 172.21.15.107:0/2421294388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92881a92a0 con 0x7f9288113190 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928ebff640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113b10 0x7f92881a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f928dbfd640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288113190 0x7f92881a3730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f927800b3f0 tx=0x7f927800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278015020 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92881a9590 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9278011c80 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278010b70 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.200+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92881afcd0 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.201+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928810db60 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.202+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f9278018020 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.202+0000 7f927f7fe640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f927407b150 0x7f927407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.203+0000 7f928e3fe640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f927407b150 0x7f927407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.203+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f927809dec0 con 0x7f9288113190 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.203+0000 7f928e3fe640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f927407b150 0x7f927407d610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92840096c0 tx=0x7f9284005a20 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.207+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f927801f020 con 0x7f9288113190 2024-09-21T11:54:17.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.391+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f92881aff90 con 0x7f9288113190 2024-09-21T11:54:17.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.392+0000 7f927f7fe640 1 -- 172.21.15.107:0/2421294388 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9278068fd0 con 0x7f9288113190 2024-09-21T11:54:17.394 INFO:teuthology.orchestra.run.smithi107.stdout:34359738420 2024-09-21T11:54:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f927407b150 msgr2=0x7f927407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f927407b150 0x7f927407d610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92840096c0 tx=0x7f9284005a20 comp rx=0 tx=0).stop 2024-09-21T11:54:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288113190 msgr2=0x7f92881a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288113190 0x7f92881a3730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f927800b3f0 tx=0x7f927800b8b0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 shutdown_connections 2024-09-21T11:54:17.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f927407b150 0x7f927407d610 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9288113b10 0x7f92881a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9288113190 0x7f92881a3730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 --2- 172.21.15.107:0/2421294388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f928810d780 0x7f92881a31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 >> 172.21.15.107:0/2421294388 conn(0x7f928806dc80 msgr2=0x7f928810a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 shutdown_connections 2024-09-21T11:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:17.394+0000 7f9290662640 1 -- 172.21.15.107:0/2421294388 wait complete. 2024-09-21T11:54:17.858 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2421294388' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T11:54:17.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2421294388' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T11:54:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2421294388' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T11:54:18.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738420 for osd.0 2024-09-21T11:54:18.128 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:18.564 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411368 for osd.2 2024-09-21T11:54:18.564 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:18.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:54:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:54:18.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:18 smithi193 ceph-mon[33333]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T11:54:18.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:18 smithi107 ceph-mon[30293]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T11:54:18.804 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:18 smithi107 ceph-mon[41097]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 -- 172.21.15.107:0/941174273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770109110 msgr2=0x7f677010f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 --2- 172.21.15.107:0/941174273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770109110 0x7f677010f910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6768009ca0 tx=0x7f676802a8c0 comp rx=0 tx=0).stop 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 -- 172.21.15.107:0/941174273 shutdown_connections 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 --2- 172.21.15.107:0/941174273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770109110 0x7f677010f910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 --2- 172.21.15.107:0/941174273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770108790 0x7f6770108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 --2- 172.21.15.107:0/941174273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770102d80 0x7f6770103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 -- 172.21.15.107:0/941174273 >> 172.21.15.107:0/941174273 conn(0x7f67700fe7d0 msgr2=0x7f6770100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.886+0000 7f677565e640 1 -- 172.21.15.107:0/941174273 shutdown_connections 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.887+0000 7f677565e640 1 -- 172.21.15.107:0/941174273 wait complete. 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.887+0000 7f677565e640 1 Processor -- start 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.887+0000 7f677565e640 1 -- start start 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f677565e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 0x7f6770198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f677565e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 0x7f6770198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f677565e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770109110 0x7f677019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f677565e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6770112260 con 0x7f6770102d80 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f677565e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67701120e0 con 0x7f6770109110 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f677565e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f67701123e0 con 0x7f6770108790 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 0x7f6770198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676e7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 0x7f6770198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676e7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 0x7f6770198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44590/0 (socket says 172.21.15.107:44590) 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 0x7f6770198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43160/0 (socket says 172.21.15.107:43160) 2024-09-21T11:54:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676e7fc640 1 -- 172.21.15.107:0/3226817320 learned_addr learned my addr 172.21.15.107:0/3226817320 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676f7fe640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770109110 0x7f677019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 -- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 msgr2=0x7f6770198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 0x7f6770198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 -- 172.21.15.107:0/3226817320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770109110 msgr2=0x7f677019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770109110 0x7f677019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 -- 172.21.15.107:0/3226817320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677019e7c0 con 0x7f6770102d80 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676e7fc640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 0x7f6770198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676f7fe640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770109110 0x7f677019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.888+0000 7f676effd640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 0x7f6770198710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f676000e9f0 tx=0x7f676000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.889+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676000cd80 con 0x7f6770102d80 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.889+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f6760004470 con 0x7f6770102d80 2024-09-21T11:54:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.889+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677019eab0 con 0x7f6770102d80 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.889+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760003e10 con 0x7f6770102d80 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.889+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67701a5170 con 0x7f6770102d80 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.890+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6770103160 con 0x7f6770102d80 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.890+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f6760003fb0 con 0x7f6770102d80 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.891+0000 7f6757fff640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f675807b150 0x7f675807d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.891+0000 7f676e7fc640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f675807b150 0x7f675807d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.891+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f676009eb30 con 0x7f6770102d80 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.891+0000 7f676e7fc640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f675807b150 0x7f675807d610 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6770108320 tx=0x7f676400a380 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:18.895+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6760069c40 con 0x7f6770102d80 2024-09-21T11:54:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.077+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f67701a5430 con 0x7f6770102d80 2024-09-21T11:54:19.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.077+0000 7f6757fff640 1 -- 172.21.15.107:0/3226817320 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f676006eaf0 con 0x7f6770102d80 2024-09-21T11:54:19.079 INFO:teuthology.orchestra.run.smithi107.stdout:214748364810 2024-09-21T11:54:19.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f675807b150 msgr2=0x7f675807d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:19.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f675807b150 0x7f675807d610 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6770108320 tx=0x7f676400a380 comp rx=0 tx=0).stop 2024-09-21T11:54:19.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 msgr2=0x7f6770198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 0x7f6770198710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f676000e9f0 tx=0x7f676000eeb0 comp rx=0 tx=0).stop 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 shutdown_connections 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f675807b150 0x7f675807d610 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6770109110 0x7f677019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6770108790 0x7f6770198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 --2- 172.21.15.107:0/3226817320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6770102d80 0x7f6770198710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.079+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 >> 172.21.15.107:0/3226817320 conn(0x7f67700fe7d0 msgr2=0x7f6770100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.080+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 shutdown_connections 2024-09-21T11:54:19.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:19.080+0000 7f677565e640 1 -- 172.21.15.107:0/3226817320 wait complete. 2024-09-21T11:54:19.306 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593743 got 184683593744 for osd.6 2024-09-21T11:54:19.307 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:19.866 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:19.866 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3226817320' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T11:54:19.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:19.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3226817320' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T11:54:19.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364808 got 214748364810 for osd.7 2024-09-21T11:54:19.896 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3226817320' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T11:54:20.346 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986203 got 133143986205 for osd.4 2024-09-21T11:54:20.346 DEBUG:teuthology.parallel:result is None 2024-09-21T11:54:20.346 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T11:54:20.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:54:20.608 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[30293]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[41097]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 -- 172.21.15.107:0/1585911788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778410d0f0 msgr2=0x7f778410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:21.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 --2- 172.21.15.107:0/1585911788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778410d0f0 0x7f778410f520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f777c009ca0 tx=0x7f777c02a8c0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 -- 172.21.15.107:0/1585911788 shutdown_connections 2024-09-21T11:54:21.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 --2- 172.21.15.107:0/1585911788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778410d0f0 0x7f778410f520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 --2- 172.21.15.107:0/1585911788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7784104540 0x7f778410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 --2- 172.21.15.107:0/1585911788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784103b40 0x7f7784103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 -- 172.21.15.107:0/1585911788 >> 172.21.15.107:0/1585911788 conn(0x7f77840fd6d0 msgr2=0x7f77840ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 -- 172.21.15.107:0/1585911788 shutdown_connections 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.152+0000 7f7789214640 1 -- 172.21.15.107:0/1585911788 wait complete. 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 Processor -- start 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 -- start start 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7784103b40 0x7f7784198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784104540 0x7f7784198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778410d0f0 0x7f778419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7784112200 con 0x7f7784103b40 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7784112080 con 0x7f7784104540 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f7789214640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7784112380 con 0x7f778410d0f0 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f778359e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778410d0f0 0x7f778419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:21.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f778259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784104540 0x7f7784198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.153+0000 7f778359e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778410d0f0 0x7f778419e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:44602/0 (socket says 172.21.15.107:44602) 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778359e640 1 -- 172.21.15.107:0/3697477318 learned_addr learned my addr 172.21.15.107:0/3697477318 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f7782d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7784103b40 0x7f7784198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778259c640 1 -- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778410d0f0 msgr2=0x7f778419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778259c640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778410d0f0 0x7f778419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778259c640 1 -- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7784103b40 msgr2=0x7f7784198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778259c640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7784103b40 0x7f7784198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778259c640 1 -- 172.21.15.107:0/3697477318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778419e800 con 0x7f7784104540 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f778259c640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784104540 0x7f7784198c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f777800e9f0 tx=0x7f777800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777800cd80 con 0x7f7784104540 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778419eaf0 con 0x7f7784104540 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7778004470 con 0x7f7784104540 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778003e10 con 0x7f7784104540 2024-09-21T11:54:21.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.154+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77841a51b0 con 0x7f7784104540 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.155+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7784103f20 con 0x7f7784104540 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.156+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f7778003fb0 con 0x7f7784104540 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.156+0000 7f776bfff640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f776c07b150 0x7f776c07d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.157+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f777809e940 con 0x7f7784104540 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.157+0000 7f7782d9d640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f776c07b150 0x7f776c07d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.157+0000 7f7782d9d640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f776c07b150 0x7f776c07d610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7774005d00 tx=0x7f777400b410 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:21.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.161+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7778069a50 con 0x7f7784104540 2024-09-21T11:54:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:20 smithi193 ceph-mon[33333]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T11:54:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:21.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.346+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 --> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7784063590 con 0x7f776c07b150 2024-09-21T11:54:21.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.354+0000 7f776bfff640 1 -- 172.21.15.107:0/3697477318 <== mgr.24491 v2:172.21.15.107:6800/271304053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346458 (secure 0 0 0) 0x7f7784063590 con 0x7f776c07b150 2024-09-21T11:54:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:54:21.361 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T11:54:21.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f776c07b150 msgr2=0x7f776c07d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:21.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f776c07b150 0x7f776c07d610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7774005d00 tx=0x7f777400b410 comp rx=0 tx=0).stop 2024-09-21T11:54:21.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784104540 msgr2=0x7f7784198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:21.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784104540 0x7f7784198c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f777800e9f0 tx=0x7f777800eeb0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 shutdown_connections 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f776c07b150 0x7f776c07d610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778410d0f0 0x7f778419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7784104540 0x7f7784198c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 --2- 172.21.15.107:0/3697477318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7784103b40 0x7f7784198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 >> 172.21.15.107:0/3697477318 conn(0x7f77840fd6d0 msgr2=0x7f77840fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 shutdown_connections 2024-09-21T11:54:21.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:21.358+0000 7f7789214640 1 -- 172.21.15.107:0/3697477318 wait complete. 2024-09-21T11:54:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi107 to 3759M 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:54:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.191 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi107 to 3759M 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.192 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.217 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-21T11:54:21.268464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244368,"kb_used_data":7356,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716848,"statfs":{"total":767960285184,"available":767710052352,"internally_reserved":0,"allocated":7532544,"data_stored":4109257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440216+0000","last_change":"2024-09-21T11:53:37.390108+0000","last_active":"2024-09-21T11:54:07.440216+0000","last_peered":"2024-09-21T11:54:07.440216+0000","last_clean":"2024-09-21T11:54:07.440216+0000","last_became_active":"2024-09-21T11:53:37.389467+0000","last_became_peered":"2024-09-21T11:53:37.389467+0000","last_unstale":"2024-09-21T11:54:07.440216+0000","last_undegraded":"2024-09-21T11:54:07.440216+0000","last_fullsized":"2024-09-21T11:54:07.440216+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:09.205261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255497+0000","last_change":"2024-09-21T11:53:39.389108+0000","last_active":"2024-09-21T11:54:07.255497+0000","last_peered":"2024-09-21T11:54:07.255497+0000","last_clean":"2024-09-21T11:54:07.255497+0000","last_became_active":"2024-09-21T11:53:39.388710+0000","last_became_peered":"2024-09-21T11:53:39.388710+0000","last_unstale":"2024-09-21T11:54:07.255497+0000","last_undegraded":"2024-09-21T11:54:07.255497+0000","last_fullsized":"2024-09-21T11:54:07.255497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:23.827988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257149+0000","last_change":"2024-09-21T11:53:41.400667+0000","last_active":"2024-09-21T11:54:07.257149+0000","last_peered":"2024-09-21T11:54:07.257149+0000","last_clean":"2024-09-21T11:54:07.257149+0000","last_became_active":"2024-09-21T11:53:41.398333+0000","last_became_peered":"2024-09-21T11:53:41.398333+0000","last_unstale":"2024-09-21T11:54:07.257149+0000","last_undegraded":"2024-09-21T11:54:07.257149+0000","last_fullsized":"2024-09-21T11:54:07.257149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:17:33.539283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255545+0000","last_change":"2024-09-21T11:53:43.398959+0000","last_active":"2024-09-21T11:54:07.255545+0000","last_peered":"2024-09-21T11:54:07.255545+0000","last_clean":"2024-09-21T11:54:07.255545+0000","last_became_active":"2024-09-21T11:53:43.398675+0000","last_became_peered":"2024-09-21T11:53:43.398675+0000","last_unstale":"2024-09-21T11:54:07.255545+0000","last_undegraded":"2024-09-21T11:54:07.255545+0000","last_fullsized":"2024-09-21T11:54:07.255545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:54.176070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439282+0000","last_change":"2024-09-21T11:53:43.442622+0000","last_active":"2024-09-21T11:54:07.439282+0000","last_peered":"2024-09-21T11:54:07.439282+0000","last_clean":"2024-09-21T11:54:07.439282+0000","last_became_active":"2024-09-21T11:53:43.442302+0000","last_became_peered":"2024-09-21T11:53:43.442302+0000","last_unstale":"2024-09-21T11:54:07.439282+0000","last_undegraded":"2024-09-21T11:54:07.439282+0000","last_fullsized":"2024-09-21T11:54:07.439282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:53.152344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.442117+0000","last_change":"2024-09-21T11:53:37.386210+0000","last_active":"2024-09-21T11:54:07.442117+0000","last_peered":"2024-09-21T11:54:07.442117+0000","last_clean":"2024-09-21T11:54:07.442117+0000","last_became_active":"2024-09-21T11:53:37.385884+0000","last_became_peered":"2024-09-21T11:53:37.385884+0000","last_unstale":"2024-09-21T11:54:07.442117+0000","last_undegraded":"2024-09-21T11:54:07.442117+0000","last_fullsized":"2024-09-21T11:54:07.442117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:54.934253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439408+0000","last_change":"2024-09-21T11:53:39.386199+0000","last_active":"2024-09-21T11:54:07.439408+0000","last_peered":"2024-09-21T11:54:07.439408+0000","last_clean":"2024-09-21T11:54:07.439408+0000","last_became_active":"2024-09-21T11:53:39.385875+0000","last_became_peered":"2024-09-21T11:53:39.385875+0000","last_unstale":"2024-09-21T11:54:07.439408+0000","last_undegraded":"2024-09-21T11:54:07.439408+0000","last_fullsized":"2024-09-21T11:54:07.439408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:01:30.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255491+0000","last_change":"2024-09-21T11:53:41.390946+0000","last_active":"2024-09-21T11:54:07.255491+0000","last_peered":"2024-09-21T11:54:07.255491+0000","last_clean":"2024-09-21T11:54:07.255491+0000","last_became_active":"2024-09-21T11:53:41.390643+0000","last_became_peered":"2024-09-21T11:53:41.390643+0000","last_unstale":"2024-09-21T11:54:07.255491+0000","last_undegraded":"2024-09-21T11:54:07.255491+0000","last_fullsized":"2024-09-21T11:54:07.255491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:14.495581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440503+0000","last_change":"2024-09-21T11:53:37.383476+0000","last_active":"2024-09-21T11:54:07.440503+0000","last_peered":"2024-09-21T11:54:07.440503+0000","last_clean":"2024-09-21T11:54:07.440503+0000","last_became_active":"2024-09-21T11:53:37.383088+0000","last_became_peered":"2024-09-21T11:53:37.383088+0000","last_unstale":"2024-09-21T11:54:07.440503+0000","last_undegraded":"2024-09-21T11:54:07.440503+0000","last_fullsized":"2024-09-21T11:54:07.440503+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:21.584368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255763+0000","last_change":"2024-09-21T11:53:39.383411+0000","last_active":"2024-09-21T11:54:07.255763+0000","last_peered":"2024-09-21T11:54:07.255763+0000","last_clean":"2024-09-21T11:54:07.255763+0000","last_became_active":"2024-09-21T11:53:39.383105+0000","last_became_peered":"2024-09-21T11:53:39.383105+0000","last_unstale":"2024-09-21T11:54:07.255763+0000","last_undegraded":"2024-09-21T11:54:07.255763+0000","last_fullsized":"2024-09-21T11:54:07.255763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:45.876015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439250+0000","last_change":"2024-09-21T11:53:41.394890+0000","last_active":"2024-09-21T11:54:07.439250+0000","last_peered":"2024-09-21T11:54:07.439250+0000","last_clean":"2024-09-21T11:54:07.439250+0000","last_became_active":"2024-09-21T11:53:41.394027+0000","last_became_peered":"2024-09-21T11:53:41.394027+0000","last_unstale":"2024-09-21T11:54:07.439250+0000","last_undegraded":"2024-09-21T11:54:07.439250+0000","last_fullsized":"2024-09-21T11:54:07.439250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.282566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440535+0000","last_change":"2024-09-21T11:53:43.398826+0000","last_active":"2024-09-21T11:54:07.440535+0000","last_peered":"2024-09-21T11:54:07.440535+0000","last_clean":"2024-09-21T11:54:07.440535+0000","last_became_active":"2024-09-21T11:53:43.398525+0000","last_became_peered":"2024-09-21T11:53:43.398525+0000","last_unstale":"2024-09-21T11:54:07.440535+0000","last_undegraded":"2024-09-21T11:54:07.440535+0000","last_fullsized":"2024-09-21T11:54:07.440535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:48.494019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439855+0000","last_change":"2024-09-21T11:53:37.383108+0000","last_active":"2024-09-21T11:54:07.439855+0000","last_peered":"2024-09-21T11:54:07.439855+0000","last_clean":"2024-09-21T11:54:07.439855+0000","last_became_active":"2024-09-21T11:53:37.382759+0000","last_became_peered":"2024-09-21T11:53:37.382759+0000","last_unstale":"2024-09-21T11:54:07.439855+0000","last_undegraded":"2024-09-21T11:54:07.439855+0000","last_fullsized":"2024-09-21T11:54:07.439855+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:08.336906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"62'5","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255864+0000","last_change":"2024-09-21T11:53:39.390871+0000","last_active":"2024-09-21T11:54:07.255864+0000","last_peered":"2024-09-21T11:54:07.255864+0000","last_clean":"2024-09-21T11:54:07.255864+0000","last_became_active":"2024-09-21T11:53:39.389678+0000","last_became_peered":"2024-09-21T11:53:39.389678+0000","last_unstale":"2024-09-21T11:54:07.255864+0000","last_undegraded":"2024-09-21T11:54:07.255864+0000","last_fullsized":"2024-09-21T11:54:07.255864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:16.441817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438678+0000","last_change":"2024-09-21T11:53:41.398756+0000","last_active":"2024-09-21T11:54:07.438678+0000","last_peered":"2024-09-21T11:54:07.438678+0000","last_clean":"2024-09-21T11:54:07.438678+0000","last_became_active":"2024-09-21T11:53:41.398485+0000","last_became_peered":"2024-09-21T11:53:41.398485+0000","last_unstale":"2024-09-21T11:54:07.438678+0000","last_undegraded":"2024-09-21T11:54:07.438678+0000","last_fullsized":"2024-09-21T11:54:07.438678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:05:22.749479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438277+0000","last_change":"2024-09-21T11:53:43.398133+0000","last_active":"2024-09-21T11:54:07.438277+0000","last_peered":"2024-09-21T11:54:07.438277+0000","last_clean":"2024-09-21T11:54:07.438277+0000","last_became_active":"2024-09-21T11:53:43.397814+0000","last_became_peered":"2024-09-21T11:53:43.397814+0000","last_unstale":"2024-09-21T11:54:07.438277+0000","last_undegraded":"2024-09-21T11:54:07.438277+0000","last_fullsized":"2024-09-21T11:54:07.438277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:18.667614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256306+0000","last_change":"2024-09-21T11:53:43.441813+0000","last_active":"2024-09-21T11:54:07.256306+0000","last_peered":"2024-09-21T11:54:07.256306+0000","last_clean":"2024-09-21T11:54:07.256306+0000","last_became_active":"2024-09-21T11:53:43.441428+0000","last_became_peered":"2024-09-21T11:53:43.441428+0000","last_unstale":"2024-09-21T11:54:07.256306+0000","last_undegraded":"2024-09-21T11:54:07.256306+0000","last_fullsized":"2024-09-21T11:54:07.256306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:58:47.409413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.442507+0000","last_change":"2024-09-21T11:53:37.382147+0000","last_active":"2024-09-21T11:54:07.442507+0000","last_peered":"2024-09-21T11:54:07.442507+0000","last_clean":"2024-09-21T11:54:07.442507+0000","last_became_active":"2024-09-21T11:53:37.381852+0000","last_became_peered":"2024-09-21T11:53:37.381852+0000","last_unstale":"2024-09-21T11:54:07.442507+0000","last_undegraded":"2024-09-21T11:54:07.442507+0000","last_fullsized":"2024-09-21T11:54:07.442507+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:00.210109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257728+0000","last_change":"2024-09-21T11:53:39.381476+0000","last_active":"2024-09-21T11:54:07.257728+0000","last_peered":"2024-09-21T11:54:07.257728+0000","last_clean":"2024-09-21T11:54:07.257728+0000","last_became_active":"2024-09-21T11:53:39.381152+0000","last_became_peered":"2024-09-21T11:53:39.381152+0000","last_unstale":"2024-09-21T11:54:07.257728+0000","last_undegraded":"2024-09-21T11:54:07.257728+0000","last_fullsized":"2024-09-21T11:54:07.257728+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:45.997786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257247+0000","last_change":"2024-09-21T11:53:41.399805+0000","last_active":"2024-09-21T11:54:07.257247+0000","last_peered":"2024-09-21T11:54:07.257247+0000","last_clean":"2024-09-21T11:54:07.257247+0000","last_became_active":"2024-09-21T11:53:41.398229+0000","last_became_peered":"2024-09-21T11:53:41.398229+0000","last_unstale":"2024-09-21T11:54:07.257247+0000","last_undegraded":"2024-09-21T11:54:07.257247+0000","last_fullsized":"2024-09-21T11:54:07.257247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:53.963380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440559+0000","last_change":"2024-09-21T11:53:43.442182+0000","last_active":"2024-09-21T11:54:07.440559+0000","last_peered":"2024-09-21T11:54:07.440559+0000","last_clean":"2024-09-21T11:54:07.440559+0000","last_became_active":"2024-09-21T11:53:43.441436+0000","last_became_peered":"2024-09-21T11:53:43.441436+0000","last_unstale":"2024-09-21T11:54:07.440559+0000","last_undegraded":"2024-09-21T11:54:07.440559+0000","last_fullsized":"2024-09-21T11:54:07.440559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:19.857928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255848+0000","last_change":"2024-09-21T11:53:37.386587+0000","last_active":"2024-09-21T11:54:07.255848+0000","last_peered":"2024-09-21T11:54:07.255848+0000","last_clean":"2024-09-21T11:54:07.255848+0000","last_became_active":"2024-09-21T11:53:37.386260+0000","last_became_peered":"2024-09-21T11:53:37.386260+0000","last_unstale":"2024-09-21T11:54:07.255848+0000","last_undegraded":"2024-09-21T11:54:07.255848+0000","last_fullsized":"2024-09-21T11:54:07.255848+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:19.595234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440188+0000","last_change":"2024-09-21T11:53:39.389100+0000","last_active":"2024-09-21T11:54:07.440188+0000","last_peered":"2024-09-21T11:54:07.440188+0000","last_clean":"2024-09-21T11:54:07.440188+0000","last_became_active":"2024-09-21T11:53:39.388817+0000","last_became_peered":"2024-09-21T11:53:39.388817+0000","last_unstale":"2024-09-21T11:54:07.440188+0000","last_undegraded":"2024-09-21T11:54:07.440188+0000","last_fullsized":"2024-09-21T11:54:07.440188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:10.083849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256107+0000","last_change":"2024-09-21T11:53:41.396514+0000","last_active":"2024-09-21T11:54:07.256107+0000","last_peered":"2024-09-21T11:54:07.256107+0000","last_clean":"2024-09-21T11:54:07.256107+0000","last_became_active":"2024-09-21T11:53:41.396245+0000","last_became_peered":"2024-09-21T11:53:41.396245+0000","last_unstale":"2024-09-21T11:54:07.256107+0000","last_undegraded":"2024-09-21T11:54:07.256107+0000","last_fullsized":"2024-09-21T11:54:07.256107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:33.029784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439413+0000","last_change":"2024-09-21T11:53:43.397014+0000","last_active":"2024-09-21T11:54:07.439413+0000","last_peered":"2024-09-21T11:54:07.439413+0000","last_clean":"2024-09-21T11:54:07.439413+0000","last_became_active":"2024-09-21T11:53:43.396697+0000","last_became_peered":"2024-09-21T11:53:43.396697+0000","last_unstale":"2024-09-21T11:54:07.439413+0000","last_undegraded":"2024-09-21T11:54:07.439413+0000","last_fullsized":"2024-09-21T11:54:07.439413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:02.497575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257810+0000","last_change":"2024-09-21T11:53:37.374074+0000","last_active":"2024-09-21T11:54:07.257810+0000","last_peered":"2024-09-21T11:54:07.257810+0000","last_clean":"2024-09-21T11:54:07.257810+0000","last_became_active":"2024-09-21T11:53:37.373476+0000","last_became_peered":"2024-09-21T11:53:37.373476+0000","last_unstale":"2024-09-21T11:54:07.257810+0000","last_undegraded":"2024-09-21T11:54:07.257810+0000","last_fullsized":"2024-09-21T11:54:07.257810+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:03.653479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441478+0000","last_change":"2024-09-21T11:53:39.388290+0000","last_active":"2024-09-21T11:54:07.441478+0000","last_peered":"2024-09-21T11:54:07.441478+0000","last_clean":"2024-09-21T11:54:07.441478+0000","last_became_active":"2024-09-21T11:53:39.387698+0000","last_became_peered":"2024-09-21T11:53:39.387698+0000","last_unstale":"2024-09-21T11:54:07.441478+0000","last_undegraded":"2024-09-21T11:54:07.441478+0000","last_fullsized":"2024-09-21T11:54:07.441478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:48.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"62'8","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438783+0000","last_change":"2024-09-21T11:53:41.392935+0000","last_active":"2024-09-21T11:54:12.438783+0000","last_peered":"2024-09-21T11:54:12.438783+0000","last_clean":"2024-09-21T11:54:12.438783+0000","last_became_active":"2024-09-21T11:53:41.392357+0000","last_became_peered":"2024-09-21T11:53:41.392357+0000","last_unstale":"2024-09-21T11:54:12.438783+0000","last_undegraded":"2024-09-21T11:54:12.438783+0000","last_fullsized":"2024-09-21T11:54:12.438783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:33.983383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257506+0000","last_change":"2024-09-21T11:53:39.383446+0000","last_active":"2024-09-21T11:54:07.257506+0000","last_peered":"2024-09-21T11:54:07.257506+0000","last_clean":"2024-09-21T11:54:07.257506+0000","last_became_active":"2024-09-21T11:53:39.383034+0000","last_became_peered":"2024-09-21T11:53:39.383034+0000","last_unstale":"2024-09-21T11:54:07.257506+0000","last_undegraded":"2024-09-21T11:54:07.257506+0000","last_fullsized":"2024-09-21T11:54:07.257506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:42.004567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438962+0000","last_change":"2024-09-21T11:53:37.385444+0000","last_active":"2024-09-21T11:54:07.438962+0000","last_peered":"2024-09-21T11:54:07.438962+0000","last_clean":"2024-09-21T11:54:07.438962+0000","last_became_active":"2024-09-21T11:53:37.383459+0000","last_became_peered":"2024-09-21T11:53:37.383459+0000","last_unstale":"2024-09-21T11:54:07.438962+0000","last_undegraded":"2024-09-21T11:54:07.438962+0000","last_fullsized":"2024-09-21T11:54:07.438962+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:43:44.381491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438783+0000","last_change":"2024-09-21T11:53:41.393810+0000","last_active":"2024-09-21T11:54:12.438783+0000","last_peered":"2024-09-21T11:54:12.438783+0000","last_clean":"2024-09-21T11:54:12.438783+0000","last_became_active":"2024-09-21T11:53:41.393517+0000","last_became_peered":"2024-09-21T11:53:41.393517+0000","last_unstale":"2024-09-21T11:54:12.438783+0000","last_undegraded":"2024-09-21T11:54:12.438783+0000","last_fullsized":"2024-09-21T11:54:12.438783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:50.172148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440194+0000","last_change":"2024-09-21T11:53:43.397504+0000","last_active":"2024-09-21T11:54:07.440194+0000","last_peered":"2024-09-21T11:54:07.440194+0000","last_clean":"2024-09-21T11:54:07.440194+0000","last_became_active":"2024-09-21T11:53:43.397211+0000","last_became_peered":"2024-09-21T11:53:43.397211+0000","last_unstale":"2024-09-21T11:54:07.440194+0000","last_undegraded":"2024-09-21T11:54:07.440194+0000","last_fullsized":"2024-09-21T11:54:07.440194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:35.051536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"62'18","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438828+0000","last_change":"2024-09-21T11:53:39.385844+0000","last_active":"2024-09-21T11:54:07.438828+0000","last_peered":"2024-09-21T11:54:07.438828+0000","last_clean":"2024-09-21T11:54:07.438828+0000","last_became_active":"2024-09-21T11:53:39.385520+0000","last_became_peered":"2024-09-21T11:53:39.385520+0000","last_unstale":"2024-09-21T11:54:07.438828+0000","last_undegraded":"2024-09-21T11:54:07.438828+0000","last_fullsized":"2024-09-21T11:54:07.438828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:27.080029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440730+0000","last_change":"2024-09-21T11:53:37.379532+0000","last_active":"2024-09-21T11:54:07.440730+0000","last_peered":"2024-09-21T11:54:07.440730+0000","last_clean":"2024-09-21T11:54:07.440730+0000","last_became_active":"2024-09-21T11:53:37.379253+0000","last_became_peered":"2024-09-21T11:53:37.379253+0000","last_unstale":"2024-09-21T11:54:07.440730+0000","last_undegraded":"2024-09-21T11:54:07.440730+0000","last_fullsized":"2024-09-21T11:54:07.440730+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:14.834964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255506+0000","last_change":"2024-09-21T11:53:41.392543+0000","last_active":"2024-09-21T11:54:07.255506+0000","last_peered":"2024-09-21T11:54:07.255506+0000","last_clean":"2024-09-21T11:54:07.255506+0000","last_became_active":"2024-09-21T11:53:41.392263+0000","last_became_peered":"2024-09-21T11:53:41.392263+0000","last_unstale":"2024-09-21T11:54:07.255506+0000","last_undegraded":"2024-09-21T11:54:07.255506+0000","last_fullsized":"2024-09-21T11:54:07.255506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:39.155714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255696+0000","last_change":"2024-09-21T11:53:43.400281+0000","last_active":"2024-09-21T11:54:07.255696+0000","last_peered":"2024-09-21T11:54:07.255696+0000","last_clean":"2024-09-21T11:54:07.255696+0000","last_became_active":"2024-09-21T11:53:43.399645+0000","last_became_peered":"2024-09-21T11:53:43.399645+0000","last_unstale":"2024-09-21T11:54:07.255696+0000","last_undegraded":"2024-09-21T11:54:07.255696+0000","last_fullsized":"2024-09-21T11:54:07.255696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:02.258139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"62'14","reported_seq":47,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255537+0000","last_change":"2024-09-21T11:53:39.391465+0000","last_active":"2024-09-21T11:54:07.255537+0000","last_peered":"2024-09-21T11:54:07.255537+0000","last_clean":"2024-09-21T11:54:07.255537+0000","last_became_active":"2024-09-21T11:53:39.389564+0000","last_became_peered":"2024-09-21T11:53:39.389564+0000","last_unstale":"2024-09-21T11:54:07.255537+0000","last_undegraded":"2024-09-21T11:54:07.255537+0000","last_fullsized":"2024-09-21T11:54:07.255537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:10.496693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441631+0000","last_change":"2024-09-21T11:53:37.381075+0000","last_active":"2024-09-21T11:54:07.441631+0000","last_peered":"2024-09-21T11:54:07.441631+0000","last_clean":"2024-09-21T11:54:07.441631+0000","last_became_active":"2024-09-21T11:53:37.380631+0000","last_became_peered":"2024-09-21T11:53:37.380631+0000","last_unstale":"2024-09-21T11:54:07.441631+0000","last_undegraded":"2024-09-21T11:54:07.441631+0000","last_fullsized":"2024-09-21T11:54:07.441631+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:36:07.721662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438450+0000","last_change":"2024-09-21T11:53:41.391662+0000","last_active":"2024-09-21T11:54:12.438450+0000","last_peered":"2024-09-21T11:54:12.438450+0000","last_clean":"2024-09-21T11:54:12.438450+0000","last_became_active":"2024-09-21T11:53:41.391333+0000","last_became_peered":"2024-09-21T11:53:41.391333+0000","last_unstale":"2024-09-21T11:54:12.438450+0000","last_undegraded":"2024-09-21T11:54:12.438450+0000","last_fullsized":"2024-09-21T11:54:12.438450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:20.273123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439539+0000","last_change":"2024-09-21T11:53:43.441782+0000","last_active":"2024-09-21T11:54:07.439539+0000","last_peered":"2024-09-21T11:54:07.439539+0000","last_clean":"2024-09-21T11:54:07.439539+0000","last_became_active":"2024-09-21T11:53:43.441363+0000","last_became_peered":"2024-09-21T11:53:43.441363+0000","last_unstale":"2024-09-21T11:54:07.439539+0000","last_undegraded":"2024-09-21T11:54:07.439539+0000","last_fullsized":"2024-09-21T11:54:07.439539+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:06:53.181853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255912+0000","last_change":"2024-09-21T11:53:39.386940+0000","last_active":"2024-09-21T11:54:07.255912+0000","last_peered":"2024-09-21T11:54:07.255912+0000","last_clean":"2024-09-21T11:54:07.255912+0000","last_became_active":"2024-09-21T11:53:39.386440+0000","last_became_peered":"2024-09-21T11:53:39.386440+0000","last_unstale":"2024-09-21T11:54:07.255912+0000","last_undegraded":"2024-09-21T11:54:07.255912+0000","last_fullsized":"2024-09-21T11:54:07.255912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:51:24.212017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440610+0000","last_change":"2024-09-21T11:53:37.385345+0000","last_active":"2024-09-21T11:54:07.440610+0000","last_peered":"2024-09-21T11:54:07.440610+0000","last_clean":"2024-09-21T11:54:07.440610+0000","last_became_active":"2024-09-21T11:53:37.384772+0000","last_became_peered":"2024-09-21T11:53:37.384772+0000","last_unstale":"2024-09-21T11:54:07.440610+0000","last_undegraded":"2024-09-21T11:54:07.440610+0000","last_fullsized":"2024-09-21T11:54:07.440610+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:45.825188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438528+0000","last_change":"2024-09-21T11:53:41.392694+0000","last_active":"2024-09-21T11:54:12.438528+0000","last_peered":"2024-09-21T11:54:12.438528+0000","last_clean":"2024-09-21T11:54:12.438528+0000","last_became_active":"2024-09-21T11:53:41.392373+0000","last_became_peered":"2024-09-21T11:53:41.392373+0000","last_unstale":"2024-09-21T11:54:12.438528+0000","last_undegraded":"2024-09-21T11:54:12.438528+0000","last_fullsized":"2024-09-21T11:54:12.438528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:03:32.290364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440416+0000","last_change":"2024-09-21T11:53:43.397497+0000","last_active":"2024-09-21T11:54:07.440416+0000","last_peered":"2024-09-21T11:54:07.440416+0000","last_clean":"2024-09-21T11:54:07.440416+0000","last_became_active":"2024-09-21T11:53:43.397188+0000","last_became_peered":"2024-09-21T11:53:43.397188+0000","last_unstale":"2024-09-21T11:54:07.440416+0000","last_undegraded":"2024-09-21T11:54:07.440416+0000","last_fullsized":"2024-09-21T11:54:07.440416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:22.203877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"62'19","reported_seq":62,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.442021+0000","last_change":"2024-09-21T11:53:39.385576+0000","last_active":"2024-09-21T11:54:07.442021+0000","last_peered":"2024-09-21T11:54:07.442021+0000","last_clean":"2024-09-21T11:54:07.442021+0000","last_became_active":"2024-09-21T11:53:39.385305+0000","last_became_peered":"2024-09-21T11:53:39.385305+0000","last_unstale":"2024-09-21T11:54:07.442021+0000","last_undegraded":"2024-09-21T11:54:07.442021+0000","last_fullsized":"2024-09-21T11:54:07.442021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:22:49.635162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257286+0000","last_change":"2024-09-21T11:53:37.386208+0000","last_active":"2024-09-21T11:54:07.257286+0000","last_peered":"2024-09-21T11:54:07.257286+0000","last_clean":"2024-09-21T11:54:07.257286+0000","last_became_active":"2024-09-21T11:53:37.385936+0000","last_became_peered":"2024-09-21T11:53:37.385936+0000","last_unstale":"2024-09-21T11:54:07.257286+0000","last_undegraded":"2024-09-21T11:54:07.257286+0000","last_fullsized":"2024-09-21T11:54:07.257286+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:44.688136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439865+0000","last_change":"2024-09-21T11:53:41.396814+0000","last_active":"2024-09-21T11:54:07.439865+0000","last_peered":"2024-09-21T11:54:07.439865+0000","last_clean":"2024-09-21T11:54:07.439865+0000","last_became_active":"2024-09-21T11:53:41.396503+0000","last_became_peered":"2024-09-21T11:53:41.396503+0000","last_unstale":"2024-09-21T11:54:07.439865+0000","last_undegraded":"2024-09-21T11:54:07.439865+0000","last_fullsized":"2024-09-21T11:54:07.439865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:30.717408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256785+0000","last_change":"2024-09-21T11:53:43.441629+0000","last_active":"2024-09-21T11:54:07.256785+0000","last_peered":"2024-09-21T11:54:07.256785+0000","last_clean":"2024-09-21T11:54:07.256785+0000","last_became_active":"2024-09-21T11:53:43.441237+0000","last_became_peered":"2024-09-21T11:53:43.441237+0000","last_unstale":"2024-09-21T11:54:07.256785+0000","last_undegraded":"2024-09-21T11:54:07.256785+0000","last_fullsized":"2024-09-21T11:54:07.256785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:14.547154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"62'28","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438476+0000","last_change":"2024-09-21T11:53:39.381612+0000","last_active":"2024-09-21T11:54:12.438476+0000","last_peered":"2024-09-21T11:54:12.438476+0000","last_clean":"2024-09-21T11:54:12.438476+0000","last_became_active":"2024-09-21T11:53:39.381281+0000","last_became_peered":"2024-09-21T11:53:39.381281+0000","last_unstale":"2024-09-21T11:54:12.438476+0000","last_undegraded":"2024-09-21T11:54:12.438476+0000","last_fullsized":"2024-09-21T11:54:12.438476+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:26.030712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256107+0000","last_change":"2024-09-21T11:53:37.382098+0000","last_active":"2024-09-21T11:54:07.256107+0000","last_peered":"2024-09-21T11:54:07.256107+0000","last_clean":"2024-09-21T11:54:07.256107+0000","last_became_active":"2024-09-21T11:53:37.381767+0000","last_became_peered":"2024-09-21T11:53:37.381767+0000","last_unstale":"2024-09-21T11:54:07.256107+0000","last_undegraded":"2024-09-21T11:54:07.256107+0000","last_fullsized":"2024-09-21T11:54:07.256107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:02.499872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"57'2","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439016+0000","last_change":"2024-09-21T11:53:39.380847+0000","last_active":"2024-09-21T11:54:07.439016+0000","last_peered":"2024-09-21T11:54:07.439016+0000","last_clean":"2024-09-21T11:54:07.439016+0000","last_became_active":"2024-09-21T11:53:36.370059+0000","last_became_peered":"2024-09-21T11:53:36.370059+0000","last_unstale":"2024-09-21T11:54:07.439016+0000","last_undegraded":"2024-09-21T11:54:07.439016+0000","last_fullsized":"2024-09-21T11:54:07.439016+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_clean_scrub_stamp":"2024-09-21T11:53:35.350854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:32.151477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031973729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439851+0000","last_change":"2024-09-21T11:53:41.401952+0000","last_active":"2024-09-21T11:54:07.439851+0000","last_peered":"2024-09-21T11:54:07.439851+0000","last_clean":"2024-09-21T11:54:07.439851+0000","last_became_active":"2024-09-21T11:53:41.401330+0000","last_became_peered":"2024-09-21T11:53:41.401330+0000","last_unstale":"2024-09-21T11:54:07.439851+0000","last_undegraded":"2024-09-21T11:54:07.439851+0000","last_fullsized":"2024-09-21T11:54:07.439851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:30.269938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441781+0000","last_change":"2024-09-21T11:53:43.399747+0000","last_active":"2024-09-21T11:54:07.441781+0000","last_peered":"2024-09-21T11:54:07.441781+0000","last_clean":"2024-09-21T11:54:07.441781+0000","last_became_active":"2024-09-21T11:53:43.399418+0000","last_became_peered":"2024-09-21T11:53:43.399418+0000","last_unstale":"2024-09-21T11:54:07.441781+0000","last_undegraded":"2024-09-21T11:54:07.441781+0000","last_fullsized":"2024-09-21T11:54:07.441781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:16:11.911786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"62'13","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257900+0000","last_change":"2024-09-21T11:53:39.384827+0000","last_active":"2024-09-21T11:54:07.257900+0000","last_peered":"2024-09-21T11:54:07.257900+0000","last_clean":"2024-09-21T11:54:07.257900+0000","last_became_active":"2024-09-21T11:53:39.384485+0000","last_became_peered":"2024-09-21T11:53:39.384485+0000","last_unstale":"2024-09-21T11:54:07.257900+0000","last_undegraded":"2024-09-21T11:54:07.257900+0000","last_fullsized":"2024-09-21T11:54:07.257900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:27.393895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256056+0000","last_change":"2024-09-21T11:53:37.385183+0000","last_active":"2024-09-21T11:54:07.256056+0000","last_peered":"2024-09-21T11:54:07.256056+0000","last_clean":"2024-09-21T11:54:07.256056+0000","last_became_active":"2024-09-21T11:53:37.384920+0000","last_became_peered":"2024-09-21T11:53:37.384920+0000","last_unstale":"2024-09-21T11:54:07.256056+0000","last_undegraded":"2024-09-21T11:54:07.256056+0000","last_fullsized":"2024-09-21T11:54:07.256056+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:18:30.851852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"55'1","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256416+0000","last_change":"2024-09-21T11:53:39.373241+0000","last_active":"2024-09-21T11:54:07.256416+0000","last_peered":"2024-09-21T11:54:07.256416+0000","last_clean":"2024-09-21T11:54:07.256416+0000","last_became_active":"2024-09-21T11:53:36.371149+0000","last_became_peered":"2024-09-21T11:53:36.371149+0000","last_unstale":"2024-09-21T11:54:07.256416+0000","last_undegraded":"2024-09-21T11:54:07.256416+0000","last_fullsized":"2024-09-21T11:54:07.256416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_clean_scrub_stamp":"2024-09-21T11:53:35.350854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:22:50.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072664400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257694+0000","last_change":"2024-09-21T11:53:41.399268+0000","last_active":"2024-09-21T11:54:07.257694+0000","last_peered":"2024-09-21T11:54:07.257694+0000","last_clean":"2024-09-21T11:54:07.257694+0000","last_became_active":"2024-09-21T11:53:41.398848+0000","last_became_peered":"2024-09-21T11:53:41.398848+0000","last_unstale":"2024-09-21T11:54:07.257694+0000","last_undegraded":"2024-09-21T11:54:07.257694+0000","last_fullsized":"2024-09-21T11:54:07.257694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:46.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438896+0000","last_change":"2024-09-21T11:53:43.442758+0000","last_active":"2024-09-21T11:54:07.438896+0000","last_peered":"2024-09-21T11:54:07.438896+0000","last_clean":"2024-09-21T11:54:07.438896+0000","last_became_active":"2024-09-21T11:53:43.442113+0000","last_became_peered":"2024-09-21T11:53:43.442113+0000","last_unstale":"2024-09-21T11:54:07.438896+0000","last_undegraded":"2024-09-21T11:54:07.438896+0000","last_fullsized":"2024-09-21T11:54:07.438896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:58:46.891451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"62'12","reported_seq":44,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441482+0000","last_change":"2024-09-21T11:53:39.387996+0000","last_active":"2024-09-21T11:54:07.441482+0000","last_peered":"2024-09-21T11:54:07.441482+0000","last_clean":"2024-09-21T11:54:07.441482+0000","last_became_active":"2024-09-21T11:53:39.386835+0000","last_became_peered":"2024-09-21T11:53:39.386835+0000","last_unstale":"2024-09-21T11:54:07.441482+0000","last_undegraded":"2024-09-21T11:54:07.441482+0000","last_fullsized":"2024-09-21T11:54:07.441482+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:23.432885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439210+0000","last_change":"2024-09-21T11:53:37.380639+0000","last_active":"2024-09-21T11:54:07.439210+0000","last_peered":"2024-09-21T11:54:07.439210+0000","last_clean":"2024-09-21T11:54:07.439210+0000","last_became_active":"2024-09-21T11:53:37.380371+0000","last_became_peered":"2024-09-21T11:53:37.380371+0000","last_unstale":"2024-09-21T11:54:07.439210+0000","last_undegraded":"2024-09-21T11:54:07.439210+0000","last_fullsized":"2024-09-21T11:54:07.439210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:25.322214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":76,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:16.456672+0000","last_change":"2024-09-21T11:53:39.373005+0000","last_active":"2024-09-21T11:54:16.456672+0000","last_peered":"2024-09-21T11:54:16.456672+0000","last_clean":"2024-09-21T11:54:16.456672+0000","last_became_active":"2024-09-21T11:53:36.371288+0000","last_became_peered":"2024-09-21T11:53:36.371288+0000","last_unstale":"2024-09-21T11:54:16.456672+0000","last_undegraded":"2024-09-21T11:54:16.456672+0000","last_fullsized":"2024-09-21T11:54:16.456672+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_clean_scrub_stamp":"2024-09-21T11:53:35.350854+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:30.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335500000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440275+0000","last_change":"2024-09-21T11:53:41.394640+0000","last_active":"2024-09-21T11:54:07.440275+0000","last_peered":"2024-09-21T11:54:07.440275+0000","last_clean":"2024-09-21T11:54:07.440275+0000","last_became_active":"2024-09-21T11:53:41.394382+0000","last_became_peered":"2024-09-21T11:53:41.394382+0000","last_unstale":"2024-09-21T11:54:07.440275+0000","last_undegraded":"2024-09-21T11:54:07.440275+0000","last_fullsized":"2024-09-21T11:54:07.440275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:08.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255755+0000","last_change":"2024-09-21T11:53:43.401627+0000","last_active":"2024-09-21T11:54:07.255755+0000","last_peered":"2024-09-21T11:54:07.255755+0000","last_clean":"2024-09-21T11:54:07.255755+0000","last_became_active":"2024-09-21T11:53:43.401296+0000","last_became_peered":"2024-09-21T11:53:43.401296+0000","last_unstale":"2024-09-21T11:54:07.255755+0000","last_undegraded":"2024-09-21T11:54:07.255755+0000","last_fullsized":"2024-09-21T11:54:07.255755+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:10.869932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"62'16","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.439005+0000","last_change":"2024-09-21T11:53:39.389242+0000","last_active":"2024-09-21T11:54:12.439005+0000","last_peered":"2024-09-21T11:54:12.439005+0000","last_clean":"2024-09-21T11:54:12.439005+0000","last_became_active":"2024-09-21T11:53:39.388187+0000","last_became_peered":"2024-09-21T11:53:39.388187+0000","last_unstale":"2024-09-21T11:54:12.439005+0000","last_undegraded":"2024-09-21T11:54:12.439005+0000","last_fullsized":"2024-09-21T11:54:12.439005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:49.247680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439082+0000","last_change":"2024-09-21T11:53:37.382507+0000","last_active":"2024-09-21T11:54:07.439082+0000","last_peered":"2024-09-21T11:54:07.439082+0000","last_clean":"2024-09-21T11:54:07.439082+0000","last_became_active":"2024-09-21T11:53:37.382217+0000","last_became_peered":"2024-09-21T11:53:37.382217+0000","last_unstale":"2024-09-21T11:54:07.439082+0000","last_undegraded":"2024-09-21T11:54:07.439082+0000","last_fullsized":"2024-09-21T11:54:07.439082+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:55.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"64'79","reported_seq":67,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:09.573035+0000","last_change":"2024-09-21T11:53:29.224600+0000","last_active":"2024-09-21T11:54:09.573035+0000","last_peered":"2024-09-21T11:54:09.573035+0000","last_clean":"2024-09-21T11:54:09.573035+0000","last_became_active":"2024-09-21T11:53:29.219335+0000","last_became_peered":"2024-09-21T11:53:29.219335+0000","last_unstale":"2024-09-21T11:54:09.573035+0000","last_undegraded":"2024-09-21T11:54:09.573035+0000","last_fullsized":"2024-09-21T11:54:09.573035+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.866315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:51:05.866315+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.866315+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.997197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440225+0000","last_change":"2024-09-21T11:53:41.397371+0000","last_active":"2024-09-21T11:54:07.440225+0000","last_peered":"2024-09-21T11:54:07.440225+0000","last_clean":"2024-09-21T11:54:07.440225+0000","last_became_active":"2024-09-21T11:53:41.397072+0000","last_became_peered":"2024-09-21T11:53:41.397072+0000","last_unstale":"2024-09-21T11:54:07.440225+0000","last_undegraded":"2024-09-21T11:54:07.440225+0000","last_fullsized":"2024-09-21T11:54:07.440225+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:55:30.061674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439748+0000","last_change":"2024-09-21T11:53:43.398169+0000","last_active":"2024-09-21T11:54:07.439748+0000","last_peered":"2024-09-21T11:54:07.439748+0000","last_clean":"2024-09-21T11:54:07.439748+0000","last_became_active":"2024-09-21T11:53:43.397837+0000","last_became_peered":"2024-09-21T11:53:43.397837+0000","last_unstale":"2024-09-21T11:54:07.439748+0000","last_undegraded":"2024-09-21T11:54:07.439748+0000","last_fullsized":"2024-09-21T11:54:07.439748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:59:51.126413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256440+0000","last_change":"2024-09-21T11:53:39.390677+0000","last_active":"2024-09-21T11:54:07.256440+0000","last_peered":"2024-09-21T11:54:07.256440+0000","last_clean":"2024-09-21T11:54:07.256440+0000","last_became_active":"2024-09-21T11:53:39.388818+0000","last_became_peered":"2024-09-21T11:53:39.388818+0000","last_unstale":"2024-09-21T11:54:07.256440+0000","last_undegraded":"2024-09-21T11:54:07.256440+0000","last_fullsized":"2024-09-21T11:54:07.256440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.706560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255841+0000","last_change":"2024-09-21T11:53:37.387061+0000","last_active":"2024-09-21T11:54:07.255841+0000","last_peered":"2024-09-21T11:54:07.255841+0000","last_clean":"2024-09-21T11:54:07.255841+0000","last_became_active":"2024-09-21T11:53:37.386729+0000","last_became_peered":"2024-09-21T11:53:37.386729+0000","last_unstale":"2024-09-21T11:54:07.255841+0000","last_undegraded":"2024-09-21T11:54:07.255841+0000","last_fullsized":"2024-09-21T11:54:07.255841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:48.070914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440010+0000","last_change":"2024-09-21T11:53:41.398234+0000","last_active":"2024-09-21T11:54:07.440010+0000","last_peered":"2024-09-21T11:54:07.440010+0000","last_clean":"2024-09-21T11:54:07.440010+0000","last_became_active":"2024-09-21T11:53:41.397918+0000","last_became_peered":"2024-09-21T11:53:41.397918+0000","last_unstale":"2024-09-21T11:54:07.440010+0000","last_undegraded":"2024-09-21T11:54:07.440010+0000","last_fullsized":"2024-09-21T11:54:07.440010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:09:13.216152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439863+0000","last_change":"2024-09-21T11:53:43.441108+0000","last_active":"2024-09-21T11:54:07.439863+0000","last_peered":"2024-09-21T11:54:07.439863+0000","last_clean":"2024-09-21T11:54:07.439863+0000","last_became_active":"2024-09-21T11:53:43.440784+0000","last_became_peered":"2024-09-21T11:53:43.440784+0000","last_unstale":"2024-09-21T11:54:07.439863+0000","last_undegraded":"2024-09-21T11:54:07.439863+0000","last_fullsized":"2024-09-21T11:54:07.439863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:51:00.192160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"62'17","reported_seq":54,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256446+0000","last_change":"2024-09-21T11:53:39.386590+0000","last_active":"2024-09-21T11:54:07.256446+0000","last_peered":"2024-09-21T11:54:07.256446+0000","last_clean":"2024-09-21T11:54:07.256446+0000","last_became_active":"2024-09-21T11:53:39.386273+0000","last_became_peered":"2024-09-21T11:53:39.386273+0000","last_unstale":"2024-09-21T11:54:07.256446+0000","last_undegraded":"2024-09-21T11:54:07.256446+0000","last_fullsized":"2024-09-21T11:54:07.256446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:32.854274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440312+0000","last_change":"2024-09-21T11:53:37.380853+0000","last_active":"2024-09-21T11:54:07.440312+0000","last_peered":"2024-09-21T11:54:07.440312+0000","last_clean":"2024-09-21T11:54:07.440312+0000","last_became_active":"2024-09-21T11:53:37.380546+0000","last_became_peered":"2024-09-21T11:53:37.380546+0000","last_unstale":"2024-09-21T11:54:07.440312+0000","last_undegraded":"2024-09-21T11:54:07.440312+0000","last_fullsized":"2024-09-21T11:54:07.440312+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:22.487427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255862+0000","last_change":"2024-09-21T11:53:41.400213+0000","last_active":"2024-09-21T11:54:07.255862+0000","last_peered":"2024-09-21T11:54:07.255862+0000","last_clean":"2024-09-21T11:54:07.255862+0000","last_became_active":"2024-09-21T11:53:41.399677+0000","last_became_peered":"2024-09-21T11:53:41.399677+0000","last_unstale":"2024-09-21T11:54:07.255862+0000","last_undegraded":"2024-09-21T11:54:07.255862+0000","last_fullsized":"2024-09-21T11:54:07.255862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:35:09.784036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257566+0000","last_change":"2024-09-21T11:53:43.397956+0000","last_active":"2024-09-21T11:54:07.257566+0000","last_peered":"2024-09-21T11:54:07.257566+0000","last_clean":"2024-09-21T11:54:07.257566+0000","last_became_active":"2024-09-21T11:53:43.397604+0000","last_became_peered":"2024-09-21T11:53:43.397604+0000","last_unstale":"2024-09-21T11:54:07.257566+0000","last_undegraded":"2024-09-21T11:54:07.257566+0000","last_fullsized":"2024-09-21T11:54:07.257566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:53.012062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256132+0000","last_change":"2024-09-21T11:53:39.387225+0000","last_active":"2024-09-21T11:54:07.256132+0000","last_peered":"2024-09-21T11:54:07.256132+0000","last_clean":"2024-09-21T11:54:07.256132+0000","last_became_active":"2024-09-21T11:53:39.386059+0000","last_became_peered":"2024-09-21T11:53:39.386059+0000","last_unstale":"2024-09-21T11:54:07.256132+0000","last_undegraded":"2024-09-21T11:54:07.256132+0000","last_fullsized":"2024-09-21T11:54:07.256132+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:32:42.582680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439541+0000","last_change":"2024-09-21T11:53:37.383041+0000","last_active":"2024-09-21T11:54:07.439541+0000","last_peered":"2024-09-21T11:54:07.439541+0000","last_clean":"2024-09-21T11:54:07.439541+0000","last_became_active":"2024-09-21T11:53:37.382747+0000","last_became_peered":"2024-09-21T11:53:37.382747+0000","last_unstale":"2024-09-21T11:54:07.439541+0000","last_undegraded":"2024-09-21T11:54:07.439541+0000","last_fullsized":"2024-09-21T11:54:07.439541+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:09.032346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438430+0000","last_change":"2024-09-21T11:53:41.399244+0000","last_active":"2024-09-21T11:54:12.438430+0000","last_peered":"2024-09-21T11:54:12.438430+0000","last_clean":"2024-09-21T11:54:12.438430+0000","last_became_active":"2024-09-21T11:53:41.398929+0000","last_became_peered":"2024-09-21T11:53:41.398929+0000","last_unstale":"2024-09-21T11:54:12.438430+0000","last_undegraded":"2024-09-21T11:54:12.438430+0000","last_fullsized":"2024-09-21T11:54:12.438430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:52:56.951308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256224+0000","last_change":"2024-09-21T11:53:43.393868+0000","last_active":"2024-09-21T11:54:07.256224+0000","last_peered":"2024-09-21T11:54:07.256224+0000","last_clean":"2024-09-21T11:54:07.256224+0000","last_became_active":"2024-09-21T11:53:43.392748+0000","last_became_peered":"2024-09-21T11:53:43.392748+0000","last_unstale":"2024-09-21T11:54:07.256224+0000","last_undegraded":"2024-09-21T11:54:07.256224+0000","last_fullsized":"2024-09-21T11:54:07.256224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:40:25.325014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441903+0000","last_change":"2024-09-21T11:53:39.386285+0000","last_active":"2024-09-21T11:54:07.441903+0000","last_peered":"2024-09-21T11:54:07.441903+0000","last_clean":"2024-09-21T11:54:07.441903+0000","last_became_active":"2024-09-21T11:53:39.385986+0000","last_became_peered":"2024-09-21T11:53:39.385986+0000","last_unstale":"2024-09-21T11:54:07.441903+0000","last_undegraded":"2024-09-21T11:54:07.441903+0000","last_fullsized":"2024-09-21T11:54:07.441903+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:40.388787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439949+0000","last_change":"2024-09-21T11:53:37.385798+0000","last_active":"2024-09-21T11:54:07.439949+0000","last_peered":"2024-09-21T11:54:07.439949+0000","last_clean":"2024-09-21T11:54:07.439949+0000","last_became_active":"2024-09-21T11:53:37.384414+0000","last_became_peered":"2024-09-21T11:53:37.384414+0000","last_unstale":"2024-09-21T11:54:07.439949+0000","last_undegraded":"2024-09-21T11:54:07.439949+0000","last_fullsized":"2024-09-21T11:54:07.439949+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:29.312339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256633+0000","last_change":"2024-09-21T11:53:41.403835+0000","last_active":"2024-09-21T11:54:07.256633+0000","last_peered":"2024-09-21T11:54:07.256633+0000","last_clean":"2024-09-21T11:54:07.256633+0000","last_became_active":"2024-09-21T11:53:41.403548+0000","last_became_peered":"2024-09-21T11:53:41.403548+0000","last_unstale":"2024-09-21T11:54:07.256633+0000","last_undegraded":"2024-09-21T11:54:07.256633+0000","last_fullsized":"2024-09-21T11:54:07.256633+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:29:57.265766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438869+0000","last_change":"2024-09-21T11:53:43.396145+0000","last_active":"2024-09-21T11:54:07.438869+0000","last_peered":"2024-09-21T11:54:07.438869+0000","last_clean":"2024-09-21T11:54:07.438869+0000","last_became_active":"2024-09-21T11:53:43.395723+0000","last_became_peered":"2024-09-21T11:53:43.395723+0000","last_unstale":"2024-09-21T11:54:07.438869+0000","last_undegraded":"2024-09-21T11:54:07.438869+0000","last_fullsized":"2024-09-21T11:54:07.438869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:45.716959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"62'19","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440059+0000","last_change":"2024-09-21T11:53:39.391483+0000","last_active":"2024-09-21T11:54:07.440059+0000","last_peered":"2024-09-21T11:54:07.440059+0000","last_clean":"2024-09-21T11:54:07.440059+0000","last_became_active":"2024-09-21T11:53:39.391172+0000","last_became_peered":"2024-09-21T11:53:39.391172+0000","last_unstale":"2024-09-21T11:54:07.440059+0000","last_undegraded":"2024-09-21T11:54:07.440059+0000","last_fullsized":"2024-09-21T11:54:07.440059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:54.323155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439263+0000","last_change":"2024-09-21T11:53:37.390266+0000","last_active":"2024-09-21T11:54:07.439263+0000","last_peered":"2024-09-21T11:54:07.439263+0000","last_clean":"2024-09-21T11:54:07.439263+0000","last_became_active":"2024-09-21T11:53:37.389929+0000","last_became_peered":"2024-09-21T11:53:37.389929+0000","last_unstale":"2024-09-21T11:54:07.439263+0000","last_undegraded":"2024-09-21T11:54:07.439263+0000","last_fullsized":"2024-09-21T11:54:07.439263+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:27.832760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256875+0000","last_change":"2024-09-21T11:53:41.403299+0000","last_active":"2024-09-21T11:54:07.256875+0000","last_peered":"2024-09-21T11:54:07.256875+0000","last_clean":"2024-09-21T11:54:07.256875+0000","last_became_active":"2024-09-21T11:53:41.403020+0000","last_became_peered":"2024-09-21T11:53:41.403020+0000","last_unstale":"2024-09-21T11:54:07.256875+0000","last_undegraded":"2024-09-21T11:54:07.256875+0000","last_fullsized":"2024-09-21T11:54:07.256875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:51:52.500090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438277+0000","last_change":"2024-09-21T11:53:43.400537+0000","last_active":"2024-09-21T11:54:07.438277+0000","last_peered":"2024-09-21T11:54:07.438277+0000","last_clean":"2024-09-21T11:54:07.438277+0000","last_became_active":"2024-09-21T11:53:43.400271+0000","last_became_peered":"2024-09-21T11:53:43.400271+0000","last_unstale":"2024-09-21T11:54:07.438277+0000","last_undegraded":"2024-09-21T11:54:07.438277+0000","last_fullsized":"2024-09-21T11:54:07.438277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:38.490320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255921+0000","last_change":"2024-09-21T11:53:39.386755+0000","last_active":"2024-09-21T11:54:07.255921+0000","last_peered":"2024-09-21T11:54:07.255921+0000","last_clean":"2024-09-21T11:54:07.255921+0000","last_became_active":"2024-09-21T11:53:39.386455+0000","last_became_peered":"2024-09-21T11:53:39.386455+0000","last_unstale":"2024-09-21T11:54:07.255921+0000","last_undegraded":"2024-09-21T11:54:07.255921+0000","last_fullsized":"2024-09-21T11:54:07.255921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:11.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439913+0000","last_change":"2024-09-21T11:53:37.386547+0000","last_active":"2024-09-21T11:54:07.439913+0000","last_peered":"2024-09-21T11:54:07.439913+0000","last_clean":"2024-09-21T11:54:07.439913+0000","last_became_active":"2024-09-21T11:53:37.386121+0000","last_became_peered":"2024-09-21T11:53:37.386121+0000","last_unstale":"2024-09-21T11:54:07.439913+0000","last_undegraded":"2024-09-21T11:54:07.439913+0000","last_fullsized":"2024-09-21T11:54:07.439913+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:27.369893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256029+0000","last_change":"2024-09-21T11:53:41.402854+0000","last_active":"2024-09-21T11:54:07.256029+0000","last_peered":"2024-09-21T11:54:07.256029+0000","last_clean":"2024-09-21T11:54:07.256029+0000","last_became_active":"2024-09-21T11:53:41.402558+0000","last_became_peered":"2024-09-21T11:53:41.402558+0000","last_unstale":"2024-09-21T11:54:07.256029+0000","last_undegraded":"2024-09-21T11:54:07.256029+0000","last_fullsized":"2024-09-21T11:54:07.256029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:16.438507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438519+0000","last_change":"2024-09-21T11:53:43.402262+0000","last_active":"2024-09-21T11:54:07.438519+0000","last_peered":"2024-09-21T11:54:07.438519+0000","last_clean":"2024-09-21T11:54:07.438519+0000","last_became_active":"2024-09-21T11:53:43.401930+0000","last_became_peered":"2024-09-21T11:53:43.401930+0000","last_unstale":"2024-09-21T11:54:07.438519+0000","last_undegraded":"2024-09-21T11:54:07.438519+0000","last_fullsized":"2024-09-21T11:54:07.438519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:16.473753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439141+0000","last_change":"2024-09-21T11:53:39.384015+0000","last_active":"2024-09-21T11:54:07.439141+0000","last_peered":"2024-09-21T11:54:07.439141+0000","last_clean":"2024-09-21T11:54:07.439141+0000","last_became_active":"2024-09-21T11:53:39.383578+0000","last_became_peered":"2024-09-21T11:53:39.383578+0000","last_unstale":"2024-09-21T11:54:07.439141+0000","last_undegraded":"2024-09-21T11:54:07.439141+0000","last_fullsized":"2024-09-21T11:54:07.439141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:16.228287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"55'2","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439293+0000","last_change":"2024-09-21T11:53:37.387440+0000","last_active":"2024-09-21T11:54:07.439293+0000","last_peered":"2024-09-21T11:54:07.439293+0000","last_clean":"2024-09-21T11:54:07.439293+0000","last_became_active":"2024-09-21T11:53:37.387070+0000","last_became_peered":"2024-09-21T11:53:37.387070+0000","last_unstale":"2024-09-21T11:54:07.439293+0000","last_undegraded":"2024-09-21T11:54:07.439293+0000","last_fullsized":"2024-09-21T11:54:07.439293+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:42.938252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438808+0000","last_change":"2024-09-21T11:53:41.398931+0000","last_active":"2024-09-21T11:54:12.438808+0000","last_peered":"2024-09-21T11:54:12.438808+0000","last_clean":"2024-09-21T11:54:12.438808+0000","last_became_active":"2024-09-21T11:53:41.398656+0000","last_became_peered":"2024-09-21T11:53:41.398656+0000","last_unstale":"2024-09-21T11:54:12.438808+0000","last_undegraded":"2024-09-21T11:54:12.438808+0000","last_fullsized":"2024-09-21T11:54:12.438808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:28.845397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438706+0000","last_change":"2024-09-21T11:53:43.441111+0000","last_active":"2024-09-21T11:54:07.438706+0000","last_peered":"2024-09-21T11:54:07.438706+0000","last_clean":"2024-09-21T11:54:07.438706+0000","last_became_active":"2024-09-21T11:53:43.440821+0000","last_became_peered":"2024-09-21T11:53:43.440821+0000","last_unstale":"2024-09-21T11:54:07.438706+0000","last_undegraded":"2024-09-21T11:54:07.438706+0000","last_fullsized":"2024-09-21T11:54:07.438706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:24.397724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440865+0000","last_change":"2024-09-21T11:53:37.384903+0000","last_active":"2024-09-21T11:54:07.440865+0000","last_peered":"2024-09-21T11:54:07.440865+0000","last_clean":"2024-09-21T11:54:07.440865+0000","last_became_active":"2024-09-21T11:53:37.384592+0000","last_became_peered":"2024-09-21T11:53:37.384592+0000","last_unstale":"2024-09-21T11:54:07.440865+0000","last_undegraded":"2024-09-21T11:54:07.440865+0000","last_fullsized":"2024-09-21T11:54:07.440865+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:26.693796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"62'6","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440701+0000","last_change":"2024-09-21T11:53:39.390279+0000","last_active":"2024-09-21T11:54:07.440701+0000","last_peered":"2024-09-21T11:54:07.440701+0000","last_clean":"2024-09-21T11:54:07.440701+0000","last_became_active":"2024-09-21T11:53:39.389878+0000","last_became_peered":"2024-09-21T11:53:39.389878+0000","last_unstale":"2024-09-21T11:54:07.440701+0000","last_undegraded":"2024-09-21T11:54:07.440701+0000","last_fullsized":"2024-09-21T11:54:07.440701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:13:07.725648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439573+0000","last_change":"2024-09-21T11:53:41.392302+0000","last_active":"2024-09-21T11:54:07.439573+0000","last_peered":"2024-09-21T11:54:07.439573+0000","last_clean":"2024-09-21T11:54:07.439573+0000","last_became_active":"2024-09-21T11:53:41.392028+0000","last_became_peered":"2024-09-21T11:53:41.392028+0000","last_unstale":"2024-09-21T11:54:07.439573+0000","last_undegraded":"2024-09-21T11:54:07.439573+0000","last_fullsized":"2024-09-21T11:54:07.439573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:34:43.601518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438514+0000","last_change":"2024-09-21T11:53:43.443026+0000","last_active":"2024-09-21T11:54:07.438514+0000","last_peered":"2024-09-21T11:54:07.438514+0000","last_clean":"2024-09-21T11:54:07.438514+0000","last_became_active":"2024-09-21T11:53:43.442481+0000","last_became_peered":"2024-09-21T11:53:43.442481+0000","last_unstale":"2024-09-21T11:54:07.438514+0000","last_undegraded":"2024-09-21T11:54:07.438514+0000","last_fullsized":"2024-09-21T11:54:07.438514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:44.477080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441560+0000","last_change":"2024-09-21T11:53:39.384630+0000","last_active":"2024-09-21T11:54:07.441560+0000","last_peered":"2024-09-21T11:54:07.441560+0000","last_clean":"2024-09-21T11:54:07.441560+0000","last_became_active":"2024-09-21T11:53:39.384041+0000","last_became_peered":"2024-09-21T11:53:39.384041+0000","last_unstale":"2024-09-21T11:54:07.441560+0000","last_undegraded":"2024-09-21T11:54:07.441560+0000","last_fullsized":"2024-09-21T11:54:07.441560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:06.761186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439922+0000","last_change":"2024-09-21T11:53:37.384725+0000","last_active":"2024-09-21T11:54:07.439922+0000","last_peered":"2024-09-21T11:54:07.439922+0000","last_clean":"2024-09-21T11:54:07.439922+0000","last_became_active":"2024-09-21T11:53:37.384415+0000","last_became_peered":"2024-09-21T11:53:37.384415+0000","last_unstale":"2024-09-21T11:54:07.439922+0000","last_undegraded":"2024-09-21T11:54:07.439922+0000","last_fullsized":"2024-09-21T11:54:07.439922+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:30.878201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440610+0000","last_change":"2024-09-21T11:53:41.392596+0000","last_active":"2024-09-21T11:54:07.440610+0000","last_peered":"2024-09-21T11:54:07.440610+0000","last_clean":"2024-09-21T11:54:07.440610+0000","last_became_active":"2024-09-21T11:53:41.392260+0000","last_became_peered":"2024-09-21T11:53:41.392260+0000","last_unstale":"2024-09-21T11:54:07.440610+0000","last_undegraded":"2024-09-21T11:54:07.440610+0000","last_fullsized":"2024-09-21T11:54:07.440610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:14.028758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257369+0000","last_change":"2024-09-21T11:53:43.396305+0000","last_active":"2024-09-21T11:54:07.257369+0000","last_peered":"2024-09-21T11:54:07.257369+0000","last_clean":"2024-09-21T11:54:07.257369+0000","last_became_active":"2024-09-21T11:53:43.396007+0000","last_became_peered":"2024-09-21T11:53:43.396007+0000","last_unstale":"2024-09-21T11:54:07.257369+0000","last_undegraded":"2024-09-21T11:54:07.257369+0000","last_fullsized":"2024-09-21T11:54:07.257369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:49.615330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440493+0000","last_change":"2024-09-21T11:53:39.383700+0000","last_active":"2024-09-21T11:54:07.440493+0000","last_peered":"2024-09-21T11:54:07.440493+0000","last_clean":"2024-09-21T11:54:07.440493+0000","last_became_active":"2024-09-21T11:53:39.383340+0000","last_became_peered":"2024-09-21T11:53:39.383340+0000","last_unstale":"2024-09-21T11:54:07.440493+0000","last_undegraded":"2024-09-21T11:54:07.440493+0000","last_fullsized":"2024-09-21T11:54:07.440493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:06.692192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439506+0000","last_change":"2024-09-21T11:53:37.379487+0000","last_active":"2024-09-21T11:54:07.439506+0000","last_peered":"2024-09-21T11:54:07.439506+0000","last_clean":"2024-09-21T11:54:07.439506+0000","last_became_active":"2024-09-21T11:53:37.379165+0000","last_became_peered":"2024-09-21T11:53:37.379165+0000","last_unstale":"2024-09-21T11:54:07.439506+0000","last_undegraded":"2024-09-21T11:54:07.439506+0000","last_fullsized":"2024-09-21T11:54:07.439506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:52.614566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438723+0000","last_change":"2024-09-21T11:53:41.396552+0000","last_active":"2024-09-21T11:54:12.438723+0000","last_peered":"2024-09-21T11:54:12.438723+0000","last_clean":"2024-09-21T11:54:12.438723+0000","last_became_active":"2024-09-21T11:53:41.396215+0000","last_became_peered":"2024-09-21T11:53:41.396215+0000","last_unstale":"2024-09-21T11:54:12.438723+0000","last_undegraded":"2024-09-21T11:54:12.438723+0000","last_fullsized":"2024-09-21T11:54:12.438723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:17.331236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256659+0000","last_change":"2024-09-21T11:53:43.395899+0000","last_active":"2024-09-21T11:54:07.256659+0000","last_peered":"2024-09-21T11:54:07.256659+0000","last_clean":"2024-09-21T11:54:07.256659+0000","last_became_active":"2024-09-21T11:53:43.395607+0000","last_became_peered":"2024-09-21T11:53:43.395607+0000","last_unstale":"2024-09-21T11:54:07.256659+0000","last_undegraded":"2024-09-21T11:54:07.256659+0000","last_fullsized":"2024-09-21T11:54:07.256659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:52.180882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440141+0000","last_change":"2024-09-21T11:53:39.383371+0000","last_active":"2024-09-21T11:54:07.440141+0000","last_peered":"2024-09-21T11:54:07.440141+0000","last_clean":"2024-09-21T11:54:07.440141+0000","last_became_active":"2024-09-21T11:53:39.383062+0000","last_became_peered":"2024-09-21T11:53:39.383062+0000","last_unstale":"2024-09-21T11:54:07.440141+0000","last_undegraded":"2024-09-21T11:54:07.440141+0000","last_fullsized":"2024-09-21T11:54:07.440141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:52.294339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439830+0000","last_change":"2024-09-21T11:53:37.387643+0000","last_active":"2024-09-21T11:54:07.439830+0000","last_peered":"2024-09-21T11:54:07.439830+0000","last_clean":"2024-09-21T11:54:07.439830+0000","last_became_active":"2024-09-21T11:53:37.387318+0000","last_became_peered":"2024-09-21T11:53:37.387318+0000","last_unstale":"2024-09-21T11:54:07.439830+0000","last_undegraded":"2024-09-21T11:54:07.439830+0000","last_fullsized":"2024-09-21T11:54:07.439830+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:41.697395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"62'8","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438796+0000","last_change":"2024-09-21T11:53:41.401581+0000","last_active":"2024-09-21T11:54:12.438796+0000","last_peered":"2024-09-21T11:54:12.438796+0000","last_clean":"2024-09-21T11:54:12.438796+0000","last_became_active":"2024-09-21T11:53:41.401284+0000","last_became_peered":"2024-09-21T11:53:41.401284+0000","last_unstale":"2024-09-21T11:54:12.438796+0000","last_undegraded":"2024-09-21T11:54:12.438796+0000","last_fullsized":"2024-09-21T11:54:12.438796+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:11.901950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441160+0000","last_change":"2024-09-21T11:53:43.396255+0000","last_active":"2024-09-21T11:54:07.441160+0000","last_peered":"2024-09-21T11:54:07.441160+0000","last_clean":"2024-09-21T11:54:07.441160+0000","last_became_active":"2024-09-21T11:53:43.395938+0000","last_became_peered":"2024-09-21T11:53:43.395938+0000","last_unstale":"2024-09-21T11:54:07.441160+0000","last_undegraded":"2024-09-21T11:54:07.441160+0000","last_fullsized":"2024-09-21T11:54:07.441160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:42.292829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255946+0000","last_change":"2024-09-21T11:53:39.390832+0000","last_active":"2024-09-21T11:54:07.255946+0000","last_peered":"2024-09-21T11:54:07.255946+0000","last_clean":"2024-09-21T11:54:07.255946+0000","last_became_active":"2024-09-21T11:53:39.388530+0000","last_became_peered":"2024-09-21T11:53:39.388530+0000","last_unstale":"2024-09-21T11:54:07.255946+0000","last_undegraded":"2024-09-21T11:54:07.255946+0000","last_fullsized":"2024-09-21T11:54:07.255946+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:00.056398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256344+0000","last_change":"2024-09-21T11:53:37.383270+0000","last_active":"2024-09-21T11:54:07.256344+0000","last_peered":"2024-09-21T11:54:07.256344+0000","last_clean":"2024-09-21T11:54:07.256344+0000","last_became_active":"2024-09-21T11:53:37.382918+0000","last_became_peered":"2024-09-21T11:53:37.382918+0000","last_unstale":"2024-09-21T11:54:07.256344+0000","last_undegraded":"2024-09-21T11:54:07.256344+0000","last_fullsized":"2024-09-21T11:54:07.256344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:50.086823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255727+0000","last_change":"2024-09-21T11:53:41.390687+0000","last_active":"2024-09-21T11:54:07.255727+0000","last_peered":"2024-09-21T11:54:07.255727+0000","last_clean":"2024-09-21T11:54:07.255727+0000","last_became_active":"2024-09-21T11:53:41.390333+0000","last_became_peered":"2024-09-21T11:53:41.390333+0000","last_unstale":"2024-09-21T11:54:07.255727+0000","last_undegraded":"2024-09-21T11:54:07.255727+0000","last_fullsized":"2024-09-21T11:54:07.255727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:42.377380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439111+0000","last_change":"2024-09-21T11:53:43.399132+0000","last_active":"2024-09-21T11:54:07.439111+0000","last_peered":"2024-09-21T11:54:07.439111+0000","last_clean":"2024-09-21T11:54:07.439111+0000","last_became_active":"2024-09-21T11:53:43.398846+0000","last_became_peered":"2024-09-21T11:53:43.398846+0000","last_unstale":"2024-09-21T11:54:07.439111+0000","last_undegraded":"2024-09-21T11:54:07.439111+0000","last_fullsized":"2024-09-21T11:54:07.439111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:42.420048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256053+0000","last_change":"2024-09-21T11:53:39.387293+0000","last_active":"2024-09-21T11:54:07.256053+0000","last_peered":"2024-09-21T11:54:07.256053+0000","last_clean":"2024-09-21T11:54:07.256053+0000","last_became_active":"2024-09-21T11:53:39.386083+0000","last_became_peered":"2024-09-21T11:53:39.386083+0000","last_unstale":"2024-09-21T11:54:07.256053+0000","last_undegraded":"2024-09-21T11:54:07.256053+0000","last_fullsized":"2024-09-21T11:54:07.256053+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:17.987422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439118+0000","last_change":"2024-09-21T11:53:37.387767+0000","last_active":"2024-09-21T11:54:07.439118+0000","last_peered":"2024-09-21T11:54:07.439118+0000","last_clean":"2024-09-21T11:54:07.439118+0000","last_became_active":"2024-09-21T11:53:37.387426+0000","last_became_peered":"2024-09-21T11:53:37.387426+0000","last_unstale":"2024-09-21T11:54:07.439118+0000","last_undegraded":"2024-09-21T11:54:07.439118+0000","last_fullsized":"2024-09-21T11:54:07.439118+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:49.280376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439387+0000","last_change":"2024-09-21T11:53:41.392884+0000","last_active":"2024-09-21T11:54:07.439387+0000","last_peered":"2024-09-21T11:54:07.439387+0000","last_clean":"2024-09-21T11:54:07.439387+0000","last_became_active":"2024-09-21T11:53:41.392588+0000","last_became_peered":"2024-09-21T11:53:41.392588+0000","last_unstale":"2024-09-21T11:54:07.439387+0000","last_undegraded":"2024-09-21T11:54:07.439387+0000","last_fullsized":"2024-09-21T11:54:07.439387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.692056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440633+0000","last_change":"2024-09-21T11:53:43.399754+0000","last_active":"2024-09-21T11:54:07.440633+0000","last_peered":"2024-09-21T11:54:07.440633+0000","last_clean":"2024-09-21T11:54:07.440633+0000","last_became_active":"2024-09-21T11:53:43.399473+0000","last_became_peered":"2024-09-21T11:53:43.399473+0000","last_unstale":"2024-09-21T11:54:07.440633+0000","last_undegraded":"2024-09-21T11:54:07.440633+0000","last_fullsized":"2024-09-21T11:54:07.440633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:39.435159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439231+0000","last_change":"2024-09-21T11:53:39.387978+0000","last_active":"2024-09-21T11:54:07.439231+0000","last_peered":"2024-09-21T11:54:07.439231+0000","last_clean":"2024-09-21T11:54:07.439231+0000","last_became_active":"2024-09-21T11:53:39.387490+0000","last_became_peered":"2024-09-21T11:53:39.387490+0000","last_unstale":"2024-09-21T11:54:07.439231+0000","last_undegraded":"2024-09-21T11:54:07.439231+0000","last_fullsized":"2024-09-21T11:54:07.439231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:36.333886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439651+0000","last_change":"2024-09-21T11:53:37.386371+0000","last_active":"2024-09-21T11:54:07.439651+0000","last_peered":"2024-09-21T11:54:07.439651+0000","last_clean":"2024-09-21T11:54:07.439651+0000","last_became_active":"2024-09-21T11:53:37.385465+0000","last_became_peered":"2024-09-21T11:53:37.385465+0000","last_unstale":"2024-09-21T11:54:07.439651+0000","last_undegraded":"2024-09-21T11:54:07.439651+0000","last_fullsized":"2024-09-21T11:54:07.439651+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:07.498543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438402+0000","last_change":"2024-09-21T11:53:41.392423+0000","last_active":"2024-09-21T11:54:07.438402+0000","last_peered":"2024-09-21T11:54:07.438402+0000","last_clean":"2024-09-21T11:54:07.438402+0000","last_became_active":"2024-09-21T11:53:41.392152+0000","last_became_peered":"2024-09-21T11:53:41.392152+0000","last_unstale":"2024-09-21T11:54:07.438402+0000","last_undegraded":"2024-09-21T11:54:07.438402+0000","last_fullsized":"2024-09-21T11:54:07.438402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:08.411208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439711+0000","last_change":"2024-09-21T11:53:43.441817+0000","last_active":"2024-09-21T11:54:07.439711+0000","last_peered":"2024-09-21T11:54:07.439711+0000","last_clean":"2024-09-21T11:54:07.439711+0000","last_became_active":"2024-09-21T11:53:43.441520+0000","last_became_peered":"2024-09-21T11:53:43.441520+0000","last_unstale":"2024-09-21T11:54:07.439711+0000","last_undegraded":"2024-09-21T11:54:07.439711+0000","last_fullsized":"2024-09-21T11:54:07.439711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:52.951050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"62'4","reported_seq":32,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438751+0000","last_change":"2024-09-21T11:53:39.386728+0000","last_active":"2024-09-21T11:54:07.438751+0000","last_peered":"2024-09-21T11:54:07.438751+0000","last_clean":"2024-09-21T11:54:07.438751+0000","last_became_active":"2024-09-21T11:53:39.386460+0000","last_became_peered":"2024-09-21T11:53:39.386460+0000","last_unstale":"2024-09-21T11:54:07.438751+0000","last_undegraded":"2024-09-21T11:54:07.438751+0000","last_fullsized":"2024-09-21T11:54:07.438751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:26.881694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440301+0000","last_change":"2024-09-21T11:53:37.390304+0000","last_active":"2024-09-21T11:54:07.440301+0000","last_peered":"2024-09-21T11:54:07.440301+0000","last_clean":"2024-09-21T11:54:07.440301+0000","last_became_active":"2024-09-21T11:53:37.389841+0000","last_became_peered":"2024-09-21T11:53:37.389841+0000","last_unstale":"2024-09-21T11:54:07.440301+0000","last_undegraded":"2024-09-21T11:54:07.440301+0000","last_fullsized":"2024-09-21T11:54:07.440301+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:42.531237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438751+0000","last_change":"2024-09-21T11:53:41.397901+0000","last_active":"2024-09-21T11:54:07.438751+0000","last_peered":"2024-09-21T11:54:07.438751+0000","last_clean":"2024-09-21T11:54:07.438751+0000","last_became_active":"2024-09-21T11:53:41.396082+0000","last_became_peered":"2024-09-21T11:53:41.396082+0000","last_unstale":"2024-09-21T11:54:07.438751+0000","last_undegraded":"2024-09-21T11:54:07.438751+0000","last_fullsized":"2024-09-21T11:54:07.438751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:35:54.054554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438517+0000","last_change":"2024-09-21T11:53:43.394884+0000","last_active":"2024-09-21T11:54:07.438517+0000","last_peered":"2024-09-21T11:54:07.438517+0000","last_clean":"2024-09-21T11:54:07.438517+0000","last_became_active":"2024-09-21T11:53:43.394563+0000","last_became_peered":"2024-09-21T11:53:43.394563+0000","last_unstale":"2024-09-21T11:54:07.438517+0000","last_undegraded":"2024-09-21T11:54:07.438517+0000","last_fullsized":"2024-09-21T11:54:07.438517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:17:05.190359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256196+0000","last_change":"2024-09-21T11:53:43.398064+0000","last_active":"2024-09-21T11:54:07.256196+0000","last_peered":"2024-09-21T11:54:07.256196+0000","last_clean":"2024-09-21T11:54:07.256196+0000","last_became_active":"2024-09-21T11:53:43.397660+0000","last_became_peered":"2024-09-21T11:53:43.397660+0000","last_unstale":"2024-09-21T11:54:07.256196+0000","last_undegraded":"2024-09-21T11:54:07.256196+0000","last_fullsized":"2024-09-21T11:54:07.256196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:48.365575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438923+0000","last_change":"2024-09-21T11:53:37.378670+0000","last_active":"2024-09-21T11:54:07.438923+0000","last_peered":"2024-09-21T11:54:07.438923+0000","last_clean":"2024-09-21T11:54:07.438923+0000","last_became_active":"2024-09-21T11:53:37.378426+0000","last_became_peered":"2024-09-21T11:53:37.378426+0000","last_unstale":"2024-09-21T11:54:07.438923+0000","last_undegraded":"2024-09-21T11:54:07.438923+0000","last_fullsized":"2024-09-21T11:54:07.438923+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:16:23.530374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439481+0000","last_change":"2024-09-21T11:53:39.394468+0000","last_active":"2024-09-21T11:54:07.439481+0000","last_peered":"2024-09-21T11:54:07.439481+0000","last_clean":"2024-09-21T11:54:07.439481+0000","last_became_active":"2024-09-21T11:53:39.394153+0000","last_became_peered":"2024-09-21T11:53:39.394153+0000","last_unstale":"2024-09-21T11:54:07.439481+0000","last_undegraded":"2024-09-21T11:54:07.439481+0000","last_fullsized":"2024-09-21T11:54:07.439481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:23.957950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441321+0000","last_change":"2024-09-21T11:53:41.395419+0000","last_active":"2024-09-21T11:54:07.441321+0000","last_peered":"2024-09-21T11:54:07.441321+0000","last_clean":"2024-09-21T11:54:07.441321+0000","last_became_active":"2024-09-21T11:53:41.395090+0000","last_became_peered":"2024-09-21T11:53:41.395090+0000","last_unstale":"2024-09-21T11:54:07.441321+0000","last_undegraded":"2024-09-21T11:54:07.441321+0000","last_fullsized":"2024-09-21T11:54:07.441321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:50:38.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":50,"seq":214748364811,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712688,"statfs":{"total":95995035648,"available":95961792512,"internally_reserved":0,"allocated":1310720,"data_stored":882909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593746,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712712,"statfs":{"total":95995035648,"available":95961817088,"internally_reserved":0,"allocated":1286144,"data_stored":881388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789975,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717444,"statfs":{"total":95995035648,"available":95966662656,"internally_reserved":0,"allocated":696320,"data_stored":291379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986206,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31896,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":724992,"data_stored":292618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149731,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31912,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713240,"statfs":{"total":95995035648,"available":95962357760,"internally_reserved":0,"allocated":741376,"data_stored":293353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717448,"statfs":{"total":95995035648,"available":95966666752,"internally_reserved":0,"allocated":692224,"data_stored":291181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":757760,"data_stored":292805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1323008,"data_stored":883624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:54:22.222 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:54:22.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi107 to 3759M 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:22.481 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:22.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:54:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 -- 172.21.15.107:0/2750740740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0102900 msgr2=0x7f1bb0102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 --2- 172.21.15.107:0/2750740740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0102900 0x7f1bb0102ce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009900 tx=0x7f1ba002a410 comp rx=0 tx=0).stop 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 -- 172.21.15.107:0/2750740740 shutdown_connections 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 --2- 172.21.15.107:0/2750740740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0108100 0x7f1bb010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 --2- 172.21.15.107:0/2750740740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bb0107760 0x7f1bb0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 --2- 172.21.15.107:0/2750740740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0102900 0x7f1bb0102ce0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 -- 172.21.15.107:0/2750740740 >> 172.21.15.107:0/2750740740 conn(0x7f1bb0076f90 msgr2=0x7f1bb00773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 -- 172.21.15.107:0/2750740740 shutdown_connections 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 -- 172.21.15.107:0/2750740740 wait complete. 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.957+0000 7f1bb529d640 1 Processor -- start 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 -- start start 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 0x7f1bb0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bb0107760 0x7f1bb0198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0108100 0x7f1bb019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0111150 con 0x7f1bb0102900 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0110fd0 con 0x7f1bb0107760 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bb529d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb01112d0 con 0x7f1bb0108100 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 0x7f1bb0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:23.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 0x7f1bb0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43204/0 (socket says 172.21.15.107:43204) 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baf7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0108100 0x7f1bb019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 -- 172.21.15.107:0/619773095 learned_addr learned my addr 172.21.15.107:0/619773095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1bae7fc640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bb0107760 0x7f1bb0198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 -- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0108100 msgr2=0x7f1bb019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0108100 0x7f1bb019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 -- 172.21.15.107:0/619773095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bb0107760 msgr2=0x7f1bb0198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bb0107760 0x7f1bb0198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baeffd640 1 -- 172.21.15.107:0/619773095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb019e8f0 con 0x7f1bb0102900 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.958+0000 7f1baf7fe640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0108100 0x7f1bb019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.959+0000 7f1baeffd640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 0x7f1bb0198840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0005940 tx=0x7f1ba0032b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.959+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba003f030 con 0x7f1bb0102900 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.959+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb019eb80 con 0x7f1bb0102900 2024-09-21T11:54:23.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.959+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f1ba0003a40 con 0x7f1bb0102900 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.959+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0035070 con 0x7f1bb0102900 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.959+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01a52a0 con 0x7f1bb0102900 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.960+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb0102ce0 con 0x7f1bb0102900 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.960+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f1ba0042030 con 0x7f1bb0102900 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.961+0000 7f1b97fff640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f1b9807b150 0x7f1b9807d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.961+0000 7f1bae7fc640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f1b9807b150 0x7f1b9807d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.961+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f1ba00bec80 con 0x7f1bb0102900 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.961+0000 7f1bae7fc640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f1b9807b150 0x7f1b9807d610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00776d0 tx=0x7f1ba4008040 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:23.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:22.965+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ba0089d90 con 0x7f1bb0102900 2024-09-21T11:54:23.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.145+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 --> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1bb0105f20 con 0x7f1b9807b150 2024-09-21T11:54:23.147 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:54:23.147 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: Deploying daemon alertmanager.a on smithi107 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T11:54:23.148 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:54:23.149 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:54:23.149 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:23.149 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: Deploying daemon alertmanager.a on smithi107 2024-09-21T11:54:23.149 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:22 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:23.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.150+0000 7f1b97fff640 1 -- 172.21.15.107:0/619773095 <== mgr.24491 v2:172.21.15.107:6800/271304053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346458 (secure 0 0 0) 0x7f1bb0105f20 con 0x7f1b9807b150 2024-09-21T11:54:23.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:54:23.157 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T11:54:23.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.153+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f1b9807b150 msgr2=0x7f1b9807d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:23.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.153+0000 7f1bb529d640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f1b9807b150 0x7f1b9807d610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1bb00776d0 tx=0x7f1ba4008040 comp rx=0 tx=0).stop 2024-09-21T11:54:23.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.153+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 msgr2=0x7f1bb0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:23.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.153+0000 7f1bb529d640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 0x7f1bb0198840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0005940 tx=0x7f1ba0032b70 comp rx=0 tx=0).stop 2024-09-21T11:54:23.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 shutdown_connections 2024-09-21T11:54:23.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f1b9807b150 0x7f1b9807d610 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb0108100 0x7f1bb019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bb0107760 0x7f1bb0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 --2- 172.21.15.107:0/619773095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb0102900 0x7f1bb0198840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:23.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 >> 172.21.15.107:0/619773095 conn(0x7f1bb0076f90 msgr2=0x7f1bb0105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:23.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 shutdown_connections 2024-09-21T11:54:23.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:23.154+0000 7f1bb529d640 1 -- 172.21.15.107:0/619773095 wait complete. 2024-09-21T11:54:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:54:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:54:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:54:23.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T11:54:23.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: Updating smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:54:23.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: Updating smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:54:23.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:23.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: Deploying daemon alertmanager.a on smithi107 2024-09-21T11:54:23.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:22 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:24.260 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-21T11:54:21.268464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244368,"kb_used_data":7356,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716848,"statfs":{"total":767960285184,"available":767710052352,"internally_reserved":0,"allocated":7532544,"data_stored":4109257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440216+0000","last_change":"2024-09-21T11:53:37.390108+0000","last_active":"2024-09-21T11:54:07.440216+0000","last_peered":"2024-09-21T11:54:07.440216+0000","last_clean":"2024-09-21T11:54:07.440216+0000","last_became_active":"2024-09-21T11:53:37.389467+0000","last_became_peered":"2024-09-21T11:53:37.389467+0000","last_unstale":"2024-09-21T11:54:07.440216+0000","last_undegraded":"2024-09-21T11:54:07.440216+0000","last_fullsized":"2024-09-21T11:54:07.440216+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:09.205261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255497+0000","last_change":"2024-09-21T11:53:39.389108+0000","last_active":"2024-09-21T11:54:07.255497+0000","last_peered":"2024-09-21T11:54:07.255497+0000","last_clean":"2024-09-21T11:54:07.255497+0000","last_became_active":"2024-09-21T11:53:39.388710+0000","last_became_peered":"2024-09-21T11:53:39.388710+0000","last_unstale":"2024-09-21T11:54:07.255497+0000","last_undegraded":"2024-09-21T11:54:07.255497+0000","last_fullsized":"2024-09-21T11:54:07.255497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:23.827988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257149+0000","last_change":"2024-09-21T11:53:41.400667+0000","last_active":"2024-09-21T11:54:07.257149+0000","last_peered":"2024-09-21T11:54:07.257149+0000","last_clean":"2024-09-21T11:54:07.257149+0000","last_became_active":"2024-09-21T11:53:41.398333+0000","last_became_peered":"2024-09-21T11:53:41.398333+0000","last_unstale":"2024-09-21T11:54:07.257149+0000","last_undegraded":"2024-09-21T11:54:07.257149+0000","last_fullsized":"2024-09-21T11:54:07.257149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:17:33.539283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255545+0000","last_change":"2024-09-21T11:53:43.398959+0000","last_active":"2024-09-21T11:54:07.255545+0000","last_peered":"2024-09-21T11:54:07.255545+0000","last_clean":"2024-09-21T11:54:07.255545+0000","last_became_active":"2024-09-21T11:53:43.398675+0000","last_became_peered":"2024-09-21T11:53:43.398675+0000","last_unstale":"2024-09-21T11:54:07.255545+0000","last_undegraded":"2024-09-21T11:54:07.255545+0000","last_fullsized":"2024-09-21T11:54:07.255545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:54.176070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439282+0000","last_change":"2024-09-21T11:53:43.442622+0000","last_active":"2024-09-21T11:54:07.439282+0000","last_peered":"2024-09-21T11:54:07.439282+0000","last_clean":"2024-09-21T11:54:07.439282+0000","last_became_active":"2024-09-21T11:53:43.442302+0000","last_became_peered":"2024-09-21T11:53:43.442302+0000","last_unstale":"2024-09-21T11:54:07.439282+0000","last_undegraded":"2024-09-21T11:54:07.439282+0000","last_fullsized":"2024-09-21T11:54:07.439282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:53.152344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.442117+0000","last_change":"2024-09-21T11:53:37.386210+0000","last_active":"2024-09-21T11:54:07.442117+0000","last_peered":"2024-09-21T11:54:07.442117+0000","last_clean":"2024-09-21T11:54:07.442117+0000","last_became_active":"2024-09-21T11:53:37.385884+0000","last_became_peered":"2024-09-21T11:53:37.385884+0000","last_unstale":"2024-09-21T11:54:07.442117+0000","last_undegraded":"2024-09-21T11:54:07.442117+0000","last_fullsized":"2024-09-21T11:54:07.442117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:54.934253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439408+0000","last_change":"2024-09-21T11:53:39.386199+0000","last_active":"2024-09-21T11:54:07.439408+0000","last_peered":"2024-09-21T11:54:07.439408+0000","last_clean":"2024-09-21T11:54:07.439408+0000","last_became_active":"2024-09-21T11:53:39.385875+0000","last_became_peered":"2024-09-21T11:53:39.385875+0000","last_unstale":"2024-09-21T11:54:07.439408+0000","last_undegraded":"2024-09-21T11:54:07.439408+0000","last_fullsized":"2024-09-21T11:54:07.439408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:01:30.368082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255491+0000","last_change":"2024-09-21T11:53:41.390946+0000","last_active":"2024-09-21T11:54:07.255491+0000","last_peered":"2024-09-21T11:54:07.255491+0000","last_clean":"2024-09-21T11:54:07.255491+0000","last_became_active":"2024-09-21T11:53:41.390643+0000","last_became_peered":"2024-09-21T11:53:41.390643+0000","last_unstale":"2024-09-21T11:54:07.255491+0000","last_undegraded":"2024-09-21T11:54:07.255491+0000","last_fullsized":"2024-09-21T11:54:07.255491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:14.495581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440503+0000","last_change":"2024-09-21T11:53:37.383476+0000","last_active":"2024-09-21T11:54:07.440503+0000","last_peered":"2024-09-21T11:54:07.440503+0000","last_clean":"2024-09-21T11:54:07.440503+0000","last_became_active":"2024-09-21T11:53:37.383088+0000","last_became_peered":"2024-09-21T11:53:37.383088+0000","last_unstale":"2024-09-21T11:54:07.440503+0000","last_undegraded":"2024-09-21T11:54:07.440503+0000","last_fullsized":"2024-09-21T11:54:07.440503+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:21.584368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255763+0000","last_change":"2024-09-21T11:53:39.383411+0000","last_active":"2024-09-21T11:54:07.255763+0000","last_peered":"2024-09-21T11:54:07.255763+0000","last_clean":"2024-09-21T11:54:07.255763+0000","last_became_active":"2024-09-21T11:53:39.383105+0000","last_became_peered":"2024-09-21T11:53:39.383105+0000","last_unstale":"2024-09-21T11:54:07.255763+0000","last_undegraded":"2024-09-21T11:54:07.255763+0000","last_fullsized":"2024-09-21T11:54:07.255763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:45.876015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439250+0000","last_change":"2024-09-21T11:53:41.394890+0000","last_active":"2024-09-21T11:54:07.439250+0000","last_peered":"2024-09-21T11:54:07.439250+0000","last_clean":"2024-09-21T11:54:07.439250+0000","last_became_active":"2024-09-21T11:53:41.394027+0000","last_became_peered":"2024-09-21T11:53:41.394027+0000","last_unstale":"2024-09-21T11:54:07.439250+0000","last_undegraded":"2024-09-21T11:54:07.439250+0000","last_fullsized":"2024-09-21T11:54:07.439250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:50.282566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440535+0000","last_change":"2024-09-21T11:53:43.398826+0000","last_active":"2024-09-21T11:54:07.440535+0000","last_peered":"2024-09-21T11:54:07.440535+0000","last_clean":"2024-09-21T11:54:07.440535+0000","last_became_active":"2024-09-21T11:53:43.398525+0000","last_became_peered":"2024-09-21T11:53:43.398525+0000","last_unstale":"2024-09-21T11:54:07.440535+0000","last_undegraded":"2024-09-21T11:54:07.440535+0000","last_fullsized":"2024-09-21T11:54:07.440535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:48.494019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439855+0000","last_change":"2024-09-21T11:53:37.383108+0000","last_active":"2024-09-21T11:54:07.439855+0000","last_peered":"2024-09-21T11:54:07.439855+0000","last_clean":"2024-09-21T11:54:07.439855+0000","last_became_active":"2024-09-21T11:53:37.382759+0000","last_became_peered":"2024-09-21T11:53:37.382759+0000","last_unstale":"2024-09-21T11:54:07.439855+0000","last_undegraded":"2024-09-21T11:54:07.439855+0000","last_fullsized":"2024-09-21T11:54:07.439855+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:08.336906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"62'5","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255864+0000","last_change":"2024-09-21T11:53:39.390871+0000","last_active":"2024-09-21T11:54:07.255864+0000","last_peered":"2024-09-21T11:54:07.255864+0000","last_clean":"2024-09-21T11:54:07.255864+0000","last_became_active":"2024-09-21T11:53:39.389678+0000","last_became_peered":"2024-09-21T11:53:39.389678+0000","last_unstale":"2024-09-21T11:54:07.255864+0000","last_undegraded":"2024-09-21T11:54:07.255864+0000","last_fullsized":"2024-09-21T11:54:07.255864+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:16.441817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438678+0000","last_change":"2024-09-21T11:53:41.398756+0000","last_active":"2024-09-21T11:54:07.438678+0000","last_peered":"2024-09-21T11:54:07.438678+0000","last_clean":"2024-09-21T11:54:07.438678+0000","last_became_active":"2024-09-21T11:53:41.398485+0000","last_became_peered":"2024-09-21T11:53:41.398485+0000","last_unstale":"2024-09-21T11:54:07.438678+0000","last_undegraded":"2024-09-21T11:54:07.438678+0000","last_fullsized":"2024-09-21T11:54:07.438678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:05:22.749479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438277+0000","last_change":"2024-09-21T11:53:43.398133+0000","last_active":"2024-09-21T11:54:07.438277+0000","last_peered":"2024-09-21T11:54:07.438277+0000","last_clean":"2024-09-21T11:54:07.438277+0000","last_became_active":"2024-09-21T11:53:43.397814+0000","last_became_peered":"2024-09-21T11:53:43.397814+0000","last_unstale":"2024-09-21T11:54:07.438277+0000","last_undegraded":"2024-09-21T11:54:07.438277+0000","last_fullsized":"2024-09-21T11:54:07.438277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:18.667614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256306+0000","last_change":"2024-09-21T11:53:43.441813+0000","last_active":"2024-09-21T11:54:07.256306+0000","last_peered":"2024-09-21T11:54:07.256306+0000","last_clean":"2024-09-21T11:54:07.256306+0000","last_became_active":"2024-09-21T11:53:43.441428+0000","last_became_peered":"2024-09-21T11:53:43.441428+0000","last_unstale":"2024-09-21T11:54:07.256306+0000","last_undegraded":"2024-09-21T11:54:07.256306+0000","last_fullsized":"2024-09-21T11:54:07.256306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:58:47.409413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.442507+0000","last_change":"2024-09-21T11:53:37.382147+0000","last_active":"2024-09-21T11:54:07.442507+0000","last_peered":"2024-09-21T11:54:07.442507+0000","last_clean":"2024-09-21T11:54:07.442507+0000","last_became_active":"2024-09-21T11:53:37.381852+0000","last_became_peered":"2024-09-21T11:53:37.381852+0000","last_unstale":"2024-09-21T11:54:07.442507+0000","last_undegraded":"2024-09-21T11:54:07.442507+0000","last_fullsized":"2024-09-21T11:54:07.442507+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:00.210109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257728+0000","last_change":"2024-09-21T11:53:39.381476+0000","last_active":"2024-09-21T11:54:07.257728+0000","last_peered":"2024-09-21T11:54:07.257728+0000","last_clean":"2024-09-21T11:54:07.257728+0000","last_became_active":"2024-09-21T11:53:39.381152+0000","last_became_peered":"2024-09-21T11:53:39.381152+0000","last_unstale":"2024-09-21T11:54:07.257728+0000","last_undegraded":"2024-09-21T11:54:07.257728+0000","last_fullsized":"2024-09-21T11:54:07.257728+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:45.997786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257247+0000","last_change":"2024-09-21T11:53:41.399805+0000","last_active":"2024-09-21T11:54:07.257247+0000","last_peered":"2024-09-21T11:54:07.257247+0000","last_clean":"2024-09-21T11:54:07.257247+0000","last_became_active":"2024-09-21T11:53:41.398229+0000","last_became_peered":"2024-09-21T11:53:41.398229+0000","last_unstale":"2024-09-21T11:54:07.257247+0000","last_undegraded":"2024-09-21T11:54:07.257247+0000","last_fullsized":"2024-09-21T11:54:07.257247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:53.963380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440559+0000","last_change":"2024-09-21T11:53:43.442182+0000","last_active":"2024-09-21T11:54:07.440559+0000","last_peered":"2024-09-21T11:54:07.440559+0000","last_clean":"2024-09-21T11:54:07.440559+0000","last_became_active":"2024-09-21T11:53:43.441436+0000","last_became_peered":"2024-09-21T11:53:43.441436+0000","last_unstale":"2024-09-21T11:54:07.440559+0000","last_undegraded":"2024-09-21T11:54:07.440559+0000","last_fullsized":"2024-09-21T11:54:07.440559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:19.857928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255848+0000","last_change":"2024-09-21T11:53:37.386587+0000","last_active":"2024-09-21T11:54:07.255848+0000","last_peered":"2024-09-21T11:54:07.255848+0000","last_clean":"2024-09-21T11:54:07.255848+0000","last_became_active":"2024-09-21T11:53:37.386260+0000","last_became_peered":"2024-09-21T11:53:37.386260+0000","last_unstale":"2024-09-21T11:54:07.255848+0000","last_undegraded":"2024-09-21T11:54:07.255848+0000","last_fullsized":"2024-09-21T11:54:07.255848+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:19.595234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440188+0000","last_change":"2024-09-21T11:53:39.389100+0000","last_active":"2024-09-21T11:54:07.440188+0000","last_peered":"2024-09-21T11:54:07.440188+0000","last_clean":"2024-09-21T11:54:07.440188+0000","last_became_active":"2024-09-21T11:53:39.388817+0000","last_became_peered":"2024-09-21T11:53:39.388817+0000","last_unstale":"2024-09-21T11:54:07.440188+0000","last_undegraded":"2024-09-21T11:54:07.440188+0000","last_fullsized":"2024-09-21T11:54:07.440188+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:10.083849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256107+0000","last_change":"2024-09-21T11:53:41.396514+0000","last_active":"2024-09-21T11:54:07.256107+0000","last_peered":"2024-09-21T11:54:07.256107+0000","last_clean":"2024-09-21T11:54:07.256107+0000","last_became_active":"2024-09-21T11:53:41.396245+0000","last_became_peered":"2024-09-21T11:53:41.396245+0000","last_unstale":"2024-09-21T11:54:07.256107+0000","last_undegraded":"2024-09-21T11:54:07.256107+0000","last_fullsized":"2024-09-21T11:54:07.256107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:33.029784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439413+0000","last_change":"2024-09-21T11:53:43.397014+0000","last_active":"2024-09-21T11:54:07.439413+0000","last_peered":"2024-09-21T11:54:07.439413+0000","last_clean":"2024-09-21T11:54:07.439413+0000","last_became_active":"2024-09-21T11:53:43.396697+0000","last_became_peered":"2024-09-21T11:53:43.396697+0000","last_unstale":"2024-09-21T11:54:07.439413+0000","last_undegraded":"2024-09-21T11:54:07.439413+0000","last_fullsized":"2024-09-21T11:54:07.439413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:02.497575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257810+0000","last_change":"2024-09-21T11:53:37.374074+0000","last_active":"2024-09-21T11:54:07.257810+0000","last_peered":"2024-09-21T11:54:07.257810+0000","last_clean":"2024-09-21T11:54:07.257810+0000","last_became_active":"2024-09-21T11:53:37.373476+0000","last_became_peered":"2024-09-21T11:53:37.373476+0000","last_unstale":"2024-09-21T11:54:07.257810+0000","last_undegraded":"2024-09-21T11:54:07.257810+0000","last_fullsized":"2024-09-21T11:54:07.257810+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:03.653479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441478+0000","last_change":"2024-09-21T11:53:39.388290+0000","last_active":"2024-09-21T11:54:07.441478+0000","last_peered":"2024-09-21T11:54:07.441478+0000","last_clean":"2024-09-21T11:54:07.441478+0000","last_became_active":"2024-09-21T11:53:39.387698+0000","last_became_peered":"2024-09-21T11:53:39.387698+0000","last_unstale":"2024-09-21T11:54:07.441478+0000","last_undegraded":"2024-09-21T11:54:07.441478+0000","last_fullsized":"2024-09-21T11:54:07.441478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:48.524085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"62'8","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438783+0000","last_change":"2024-09-21T11:53:41.392935+0000","last_active":"2024-09-21T11:54:12.438783+0000","last_peered":"2024-09-21T11:54:12.438783+0000","last_clean":"2024-09-21T11:54:12.438783+0000","last_became_active":"2024-09-21T11:53:41.392357+0000","last_became_peered":"2024-09-21T11:53:41.392357+0000","last_unstale":"2024-09-21T11:54:12.438783+0000","last_undegraded":"2024-09-21T11:54:12.438783+0000","last_fullsized":"2024-09-21T11:54:12.438783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:33.983383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257506+0000","last_change":"2024-09-21T11:53:39.383446+0000","last_active":"2024-09-21T11:54:07.257506+0000","last_peered":"2024-09-21T11:54:07.257506+0000","last_clean":"2024-09-21T11:54:07.257506+0000","last_became_active":"2024-09-21T11:53:39.383034+0000","last_became_peered":"2024-09-21T11:53:39.383034+0000","last_unstale":"2024-09-21T11:54:07.257506+0000","last_undegraded":"2024-09-21T11:54:07.257506+0000","last_fullsized":"2024-09-21T11:54:07.257506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:42.004567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438962+0000","last_change":"2024-09-21T11:53:37.385444+0000","last_active":"2024-09-21T11:54:07.438962+0000","last_peered":"2024-09-21T11:54:07.438962+0000","last_clean":"2024-09-21T11:54:07.438962+0000","last_became_active":"2024-09-21T11:53:37.383459+0000","last_became_peered":"2024-09-21T11:53:37.383459+0000","last_unstale":"2024-09-21T11:54:07.438962+0000","last_undegraded":"2024-09-21T11:54:07.438962+0000","last_fullsized":"2024-09-21T11:54:07.438962+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:43:44.381491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438783+0000","last_change":"2024-09-21T11:53:41.393810+0000","last_active":"2024-09-21T11:54:12.438783+0000","last_peered":"2024-09-21T11:54:12.438783+0000","last_clean":"2024-09-21T11:54:12.438783+0000","last_became_active":"2024-09-21T11:53:41.393517+0000","last_became_peered":"2024-09-21T11:53:41.393517+0000","last_unstale":"2024-09-21T11:54:12.438783+0000","last_undegraded":"2024-09-21T11:54:12.438783+0000","last_fullsized":"2024-09-21T11:54:12.438783+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:50.172148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440194+0000","last_change":"2024-09-21T11:53:43.397504+0000","last_active":"2024-09-21T11:54:07.440194+0000","last_peered":"2024-09-21T11:54:07.440194+0000","last_clean":"2024-09-21T11:54:07.440194+0000","last_became_active":"2024-09-21T11:53:43.397211+0000","last_became_peered":"2024-09-21T11:53:43.397211+0000","last_unstale":"2024-09-21T11:54:07.440194+0000","last_undegraded":"2024-09-21T11:54:07.440194+0000","last_fullsized":"2024-09-21T11:54:07.440194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:35.051536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"62'18","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438828+0000","last_change":"2024-09-21T11:53:39.385844+0000","last_active":"2024-09-21T11:54:07.438828+0000","last_peered":"2024-09-21T11:54:07.438828+0000","last_clean":"2024-09-21T11:54:07.438828+0000","last_became_active":"2024-09-21T11:53:39.385520+0000","last_became_peered":"2024-09-21T11:53:39.385520+0000","last_unstale":"2024-09-21T11:54:07.438828+0000","last_undegraded":"2024-09-21T11:54:07.438828+0000","last_fullsized":"2024-09-21T11:54:07.438828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:27.080029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440730+0000","last_change":"2024-09-21T11:53:37.379532+0000","last_active":"2024-09-21T11:54:07.440730+0000","last_peered":"2024-09-21T11:54:07.440730+0000","last_clean":"2024-09-21T11:54:07.440730+0000","last_became_active":"2024-09-21T11:53:37.379253+0000","last_became_peered":"2024-09-21T11:53:37.379253+0000","last_unstale":"2024-09-21T11:54:07.440730+0000","last_undegraded":"2024-09-21T11:54:07.440730+0000","last_fullsized":"2024-09-21T11:54:07.440730+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:14.834964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255506+0000","last_change":"2024-09-21T11:53:41.392543+0000","last_active":"2024-09-21T11:54:07.255506+0000","last_peered":"2024-09-21T11:54:07.255506+0000","last_clean":"2024-09-21T11:54:07.255506+0000","last_became_active":"2024-09-21T11:53:41.392263+0000","last_became_peered":"2024-09-21T11:53:41.392263+0000","last_unstale":"2024-09-21T11:54:07.255506+0000","last_undegraded":"2024-09-21T11:54:07.255506+0000","last_fullsized":"2024-09-21T11:54:07.255506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:39.155714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255696+0000","last_change":"2024-09-21T11:53:43.400281+0000","last_active":"2024-09-21T11:54:07.255696+0000","last_peered":"2024-09-21T11:54:07.255696+0000","last_clean":"2024-09-21T11:54:07.255696+0000","last_became_active":"2024-09-21T11:53:43.399645+0000","last_became_peered":"2024-09-21T11:53:43.399645+0000","last_unstale":"2024-09-21T11:54:07.255696+0000","last_undegraded":"2024-09-21T11:54:07.255696+0000","last_fullsized":"2024-09-21T11:54:07.255696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:02.258139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"62'14","reported_seq":47,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255537+0000","last_change":"2024-09-21T11:53:39.391465+0000","last_active":"2024-09-21T11:54:07.255537+0000","last_peered":"2024-09-21T11:54:07.255537+0000","last_clean":"2024-09-21T11:54:07.255537+0000","last_became_active":"2024-09-21T11:53:39.389564+0000","last_became_peered":"2024-09-21T11:53:39.389564+0000","last_unstale":"2024-09-21T11:54:07.255537+0000","last_undegraded":"2024-09-21T11:54:07.255537+0000","last_fullsized":"2024-09-21T11:54:07.255537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:10.496693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"55'1","reported_seq":31,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441631+0000","last_change":"2024-09-21T11:53:37.381075+0000","last_active":"2024-09-21T11:54:07.441631+0000","last_peered":"2024-09-21T11:54:07.441631+0000","last_clean":"2024-09-21T11:54:07.441631+0000","last_became_active":"2024-09-21T11:53:37.380631+0000","last_became_peered":"2024-09-21T11:53:37.380631+0000","last_unstale":"2024-09-21T11:54:07.441631+0000","last_undegraded":"2024-09-21T11:54:07.441631+0000","last_fullsized":"2024-09-21T11:54:07.441631+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:36:07.721662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438450+0000","last_change":"2024-09-21T11:53:41.391662+0000","last_active":"2024-09-21T11:54:12.438450+0000","last_peered":"2024-09-21T11:54:12.438450+0000","last_clean":"2024-09-21T11:54:12.438450+0000","last_became_active":"2024-09-21T11:53:41.391333+0000","last_became_peered":"2024-09-21T11:53:41.391333+0000","last_unstale":"2024-09-21T11:54:12.438450+0000","last_undegraded":"2024-09-21T11:54:12.438450+0000","last_fullsized":"2024-09-21T11:54:12.438450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:20.273123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439539+0000","last_change":"2024-09-21T11:53:43.441782+0000","last_active":"2024-09-21T11:54:07.439539+0000","last_peered":"2024-09-21T11:54:07.439539+0000","last_clean":"2024-09-21T11:54:07.439539+0000","last_became_active":"2024-09-21T11:53:43.441363+0000","last_became_peered":"2024-09-21T11:53:43.441363+0000","last_unstale":"2024-09-21T11:54:07.439539+0000","last_undegraded":"2024-09-21T11:54:07.439539+0000","last_fullsized":"2024-09-21T11:54:07.439539+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:06:53.181853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255912+0000","last_change":"2024-09-21T11:53:39.386940+0000","last_active":"2024-09-21T11:54:07.255912+0000","last_peered":"2024-09-21T11:54:07.255912+0000","last_clean":"2024-09-21T11:54:07.255912+0000","last_became_active":"2024-09-21T11:53:39.386440+0000","last_became_peered":"2024-09-21T11:53:39.386440+0000","last_unstale":"2024-09-21T11:54:07.255912+0000","last_undegraded":"2024-09-21T11:54:07.255912+0000","last_fullsized":"2024-09-21T11:54:07.255912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:51:24.212017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440610+0000","last_change":"2024-09-21T11:53:37.385345+0000","last_active":"2024-09-21T11:54:07.440610+0000","last_peered":"2024-09-21T11:54:07.440610+0000","last_clean":"2024-09-21T11:54:07.440610+0000","last_became_active":"2024-09-21T11:53:37.384772+0000","last_became_peered":"2024-09-21T11:53:37.384772+0000","last_unstale":"2024-09-21T11:54:07.440610+0000","last_undegraded":"2024-09-21T11:54:07.440610+0000","last_fullsized":"2024-09-21T11:54:07.440610+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:45.825188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438528+0000","last_change":"2024-09-21T11:53:41.392694+0000","last_active":"2024-09-21T11:54:12.438528+0000","last_peered":"2024-09-21T11:54:12.438528+0000","last_clean":"2024-09-21T11:54:12.438528+0000","last_became_active":"2024-09-21T11:53:41.392373+0000","last_became_peered":"2024-09-21T11:53:41.392373+0000","last_unstale":"2024-09-21T11:54:12.438528+0000","last_undegraded":"2024-09-21T11:54:12.438528+0000","last_fullsized":"2024-09-21T11:54:12.438528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:03:32.290364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"62'1","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440416+0000","last_change":"2024-09-21T11:53:43.397497+0000","last_active":"2024-09-21T11:54:07.440416+0000","last_peered":"2024-09-21T11:54:07.440416+0000","last_clean":"2024-09-21T11:54:07.440416+0000","last_became_active":"2024-09-21T11:53:43.397188+0000","last_became_peered":"2024-09-21T11:53:43.397188+0000","last_unstale":"2024-09-21T11:54:07.440416+0000","last_undegraded":"2024-09-21T11:54:07.440416+0000","last_fullsized":"2024-09-21T11:54:07.440416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:22.203877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"62'19","reported_seq":62,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.442021+0000","last_change":"2024-09-21T11:53:39.385576+0000","last_active":"2024-09-21T11:54:07.442021+0000","last_peered":"2024-09-21T11:54:07.442021+0000","last_clean":"2024-09-21T11:54:07.442021+0000","last_became_active":"2024-09-21T11:53:39.385305+0000","last_became_peered":"2024-09-21T11:53:39.385305+0000","last_unstale":"2024-09-21T11:54:07.442021+0000","last_undegraded":"2024-09-21T11:54:07.442021+0000","last_fullsized":"2024-09-21T11:54:07.442021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:22:49.635162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257286+0000","last_change":"2024-09-21T11:53:37.386208+0000","last_active":"2024-09-21T11:54:07.257286+0000","last_peered":"2024-09-21T11:54:07.257286+0000","last_clean":"2024-09-21T11:54:07.257286+0000","last_became_active":"2024-09-21T11:53:37.385936+0000","last_became_peered":"2024-09-21T11:53:37.385936+0000","last_unstale":"2024-09-21T11:54:07.257286+0000","last_undegraded":"2024-09-21T11:54:07.257286+0000","last_fullsized":"2024-09-21T11:54:07.257286+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:44.688136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439865+0000","last_change":"2024-09-21T11:53:41.396814+0000","last_active":"2024-09-21T11:54:07.439865+0000","last_peered":"2024-09-21T11:54:07.439865+0000","last_clean":"2024-09-21T11:54:07.439865+0000","last_became_active":"2024-09-21T11:53:41.396503+0000","last_became_peered":"2024-09-21T11:53:41.396503+0000","last_unstale":"2024-09-21T11:54:07.439865+0000","last_undegraded":"2024-09-21T11:54:07.439865+0000","last_fullsized":"2024-09-21T11:54:07.439865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:30.717408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256785+0000","last_change":"2024-09-21T11:53:43.441629+0000","last_active":"2024-09-21T11:54:07.256785+0000","last_peered":"2024-09-21T11:54:07.256785+0000","last_clean":"2024-09-21T11:54:07.256785+0000","last_became_active":"2024-09-21T11:53:43.441237+0000","last_became_peered":"2024-09-21T11:53:43.441237+0000","last_unstale":"2024-09-21T11:54:07.256785+0000","last_undegraded":"2024-09-21T11:54:07.256785+0000","last_fullsized":"2024-09-21T11:54:07.256785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:14.547154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"62'28","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438476+0000","last_change":"2024-09-21T11:53:39.381612+0000","last_active":"2024-09-21T11:54:12.438476+0000","last_peered":"2024-09-21T11:54:12.438476+0000","last_clean":"2024-09-21T11:54:12.438476+0000","last_became_active":"2024-09-21T11:53:39.381281+0000","last_became_peered":"2024-09-21T11:53:39.381281+0000","last_unstale":"2024-09-21T11:54:12.438476+0000","last_undegraded":"2024-09-21T11:54:12.438476+0000","last_fullsized":"2024-09-21T11:54:12.438476+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:26.030712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256107+0000","last_change":"2024-09-21T11:53:37.382098+0000","last_active":"2024-09-21T11:54:07.256107+0000","last_peered":"2024-09-21T11:54:07.256107+0000","last_clean":"2024-09-21T11:54:07.256107+0000","last_became_active":"2024-09-21T11:53:37.381767+0000","last_became_peered":"2024-09-21T11:53:37.381767+0000","last_unstale":"2024-09-21T11:54:07.256107+0000","last_undegraded":"2024-09-21T11:54:07.256107+0000","last_fullsized":"2024-09-21T11:54:07.256107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:02.499872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"57'2","reported_seq":40,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439016+0000","last_change":"2024-09-21T11:53:39.380847+0000","last_active":"2024-09-21T11:54:07.439016+0000","last_peered":"2024-09-21T11:54:07.439016+0000","last_clean":"2024-09-21T11:54:07.439016+0000","last_became_active":"2024-09-21T11:53:36.370059+0000","last_became_peered":"2024-09-21T11:53:36.370059+0000","last_unstale":"2024-09-21T11:54:07.439016+0000","last_undegraded":"2024-09-21T11:54:07.439016+0000","last_fullsized":"2024-09-21T11:54:07.439016+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_clean_scrub_stamp":"2024-09-21T11:53:35.350854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:32.151477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031973729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439851+0000","last_change":"2024-09-21T11:53:41.401952+0000","last_active":"2024-09-21T11:54:07.439851+0000","last_peered":"2024-09-21T11:54:07.439851+0000","last_clean":"2024-09-21T11:54:07.439851+0000","last_became_active":"2024-09-21T11:53:41.401330+0000","last_became_peered":"2024-09-21T11:53:41.401330+0000","last_unstale":"2024-09-21T11:54:07.439851+0000","last_undegraded":"2024-09-21T11:54:07.439851+0000","last_fullsized":"2024-09-21T11:54:07.439851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:30.269938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441781+0000","last_change":"2024-09-21T11:53:43.399747+0000","last_active":"2024-09-21T11:54:07.441781+0000","last_peered":"2024-09-21T11:54:07.441781+0000","last_clean":"2024-09-21T11:54:07.441781+0000","last_became_active":"2024-09-21T11:53:43.399418+0000","last_became_peered":"2024-09-21T11:53:43.399418+0000","last_unstale":"2024-09-21T11:54:07.441781+0000","last_undegraded":"2024-09-21T11:54:07.441781+0000","last_fullsized":"2024-09-21T11:54:07.441781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:16:11.911786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"62'13","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257900+0000","last_change":"2024-09-21T11:53:39.384827+0000","last_active":"2024-09-21T11:54:07.257900+0000","last_peered":"2024-09-21T11:54:07.257900+0000","last_clean":"2024-09-21T11:54:07.257900+0000","last_became_active":"2024-09-21T11:53:39.384485+0000","last_became_peered":"2024-09-21T11:53:39.384485+0000","last_unstale":"2024-09-21T11:54:07.257900+0000","last_undegraded":"2024-09-21T11:54:07.257900+0000","last_fullsized":"2024-09-21T11:54:07.257900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:27.393895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256056+0000","last_change":"2024-09-21T11:53:37.385183+0000","last_active":"2024-09-21T11:54:07.256056+0000","last_peered":"2024-09-21T11:54:07.256056+0000","last_clean":"2024-09-21T11:54:07.256056+0000","last_became_active":"2024-09-21T11:53:37.384920+0000","last_became_peered":"2024-09-21T11:53:37.384920+0000","last_unstale":"2024-09-21T11:54:07.256056+0000","last_undegraded":"2024-09-21T11:54:07.256056+0000","last_fullsized":"2024-09-21T11:54:07.256056+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:18:30.851852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"55'1","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256416+0000","last_change":"2024-09-21T11:53:39.373241+0000","last_active":"2024-09-21T11:54:07.256416+0000","last_peered":"2024-09-21T11:54:07.256416+0000","last_clean":"2024-09-21T11:54:07.256416+0000","last_became_active":"2024-09-21T11:53:36.371149+0000","last_became_peered":"2024-09-21T11:53:36.371149+0000","last_unstale":"2024-09-21T11:54:07.256416+0000","last_undegraded":"2024-09-21T11:54:07.256416+0000","last_fullsized":"2024-09-21T11:54:07.256416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_clean_scrub_stamp":"2024-09-21T11:53:35.350854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:22:50.701615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072664400000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257694+0000","last_change":"2024-09-21T11:53:41.399268+0000","last_active":"2024-09-21T11:54:07.257694+0000","last_peered":"2024-09-21T11:54:07.257694+0000","last_clean":"2024-09-21T11:54:07.257694+0000","last_became_active":"2024-09-21T11:53:41.398848+0000","last_became_peered":"2024-09-21T11:53:41.398848+0000","last_unstale":"2024-09-21T11:54:07.257694+0000","last_undegraded":"2024-09-21T11:54:07.257694+0000","last_fullsized":"2024-09-21T11:54:07.257694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:46.205511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438896+0000","last_change":"2024-09-21T11:53:43.442758+0000","last_active":"2024-09-21T11:54:07.438896+0000","last_peered":"2024-09-21T11:54:07.438896+0000","last_clean":"2024-09-21T11:54:07.438896+0000","last_became_active":"2024-09-21T11:53:43.442113+0000","last_became_peered":"2024-09-21T11:53:43.442113+0000","last_unstale":"2024-09-21T11:54:07.438896+0000","last_undegraded":"2024-09-21T11:54:07.438896+0000","last_fullsized":"2024-09-21T11:54:07.438896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:58:46.891451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"62'12","reported_seq":44,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441482+0000","last_change":"2024-09-21T11:53:39.387996+0000","last_active":"2024-09-21T11:54:07.441482+0000","last_peered":"2024-09-21T11:54:07.441482+0000","last_clean":"2024-09-21T11:54:07.441482+0000","last_became_active":"2024-09-21T11:53:39.386835+0000","last_became_peered":"2024-09-21T11:53:39.386835+0000","last_unstale":"2024-09-21T11:54:07.441482+0000","last_undegraded":"2024-09-21T11:54:07.441482+0000","last_fullsized":"2024-09-21T11:54:07.441482+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:23.432885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":36,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439210+0000","last_change":"2024-09-21T11:53:37.380639+0000","last_active":"2024-09-21T11:54:07.439210+0000","last_peered":"2024-09-21T11:54:07.439210+0000","last_clean":"2024-09-21T11:54:07.439210+0000","last_became_active":"2024-09-21T11:53:37.380371+0000","last_became_peered":"2024-09-21T11:53:37.380371+0000","last_unstale":"2024-09-21T11:54:07.439210+0000","last_undegraded":"2024-09-21T11:54:07.439210+0000","last_fullsized":"2024-09-21T11:54:07.439210+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:25.322214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'5","reported_seq":76,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:16.456672+0000","last_change":"2024-09-21T11:53:39.373005+0000","last_active":"2024-09-21T11:54:16.456672+0000","last_peered":"2024-09-21T11:54:16.456672+0000","last_clean":"2024-09-21T11:54:16.456672+0000","last_became_active":"2024-09-21T11:53:36.371288+0000","last_became_peered":"2024-09-21T11:53:36.371288+0000","last_unstale":"2024-09-21T11:54:16.456672+0000","last_undegraded":"2024-09-21T11:54:16.456672+0000","last_fullsized":"2024-09-21T11:54:16.456672+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:35.350854+0000","last_clean_scrub_stamp":"2024-09-21T11:53:35.350854+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:30.937643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053335500000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440275+0000","last_change":"2024-09-21T11:53:41.394640+0000","last_active":"2024-09-21T11:54:07.440275+0000","last_peered":"2024-09-21T11:54:07.440275+0000","last_clean":"2024-09-21T11:54:07.440275+0000","last_became_active":"2024-09-21T11:53:41.394382+0000","last_became_peered":"2024-09-21T11:53:41.394382+0000","last_unstale":"2024-09-21T11:54:07.440275+0000","last_undegraded":"2024-09-21T11:54:07.440275+0000","last_fullsized":"2024-09-21T11:54:07.440275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:08.119851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255755+0000","last_change":"2024-09-21T11:53:43.401627+0000","last_active":"2024-09-21T11:54:07.255755+0000","last_peered":"2024-09-21T11:54:07.255755+0000","last_clean":"2024-09-21T11:54:07.255755+0000","last_became_active":"2024-09-21T11:53:43.401296+0000","last_became_peered":"2024-09-21T11:53:43.401296+0000","last_unstale":"2024-09-21T11:54:07.255755+0000","last_undegraded":"2024-09-21T11:54:07.255755+0000","last_fullsized":"2024-09-21T11:54:07.255755+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:10.869932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"62'16","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.439005+0000","last_change":"2024-09-21T11:53:39.389242+0000","last_active":"2024-09-21T11:54:12.439005+0000","last_peered":"2024-09-21T11:54:12.439005+0000","last_clean":"2024-09-21T11:54:12.439005+0000","last_became_active":"2024-09-21T11:53:39.388187+0000","last_became_peered":"2024-09-21T11:53:39.388187+0000","last_unstale":"2024-09-21T11:54:12.439005+0000","last_undegraded":"2024-09-21T11:54:12.439005+0000","last_fullsized":"2024-09-21T11:54:12.439005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:49.247680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439082+0000","last_change":"2024-09-21T11:53:37.382507+0000","last_active":"2024-09-21T11:54:07.439082+0000","last_peered":"2024-09-21T11:54:07.439082+0000","last_clean":"2024-09-21T11:54:07.439082+0000","last_became_active":"2024-09-21T11:53:37.382217+0000","last_became_peered":"2024-09-21T11:53:37.382217+0000","last_unstale":"2024-09-21T11:54:07.439082+0000","last_undegraded":"2024-09-21T11:54:07.439082+0000","last_fullsized":"2024-09-21T11:54:07.439082+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:55.398640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"64'79","reported_seq":67,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:09.573035+0000","last_change":"2024-09-21T11:53:29.224600+0000","last_active":"2024-09-21T11:54:09.573035+0000","last_peered":"2024-09-21T11:54:09.573035+0000","last_clean":"2024-09-21T11:54:09.573035+0000","last_became_active":"2024-09-21T11:53:29.219335+0000","last_became_peered":"2024-09-21T11:53:29.219335+0000","last_unstale":"2024-09-21T11:54:09.573035+0000","last_undegraded":"2024-09-21T11:54:09.573035+0000","last_fullsized":"2024-09-21T11:54:09.573035+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.866315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:51:05.866315+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.866315+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.997197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440225+0000","last_change":"2024-09-21T11:53:41.397371+0000","last_active":"2024-09-21T11:54:07.440225+0000","last_peered":"2024-09-21T11:54:07.440225+0000","last_clean":"2024-09-21T11:54:07.440225+0000","last_became_active":"2024-09-21T11:53:41.397072+0000","last_became_peered":"2024-09-21T11:53:41.397072+0000","last_unstale":"2024-09-21T11:54:07.440225+0000","last_undegraded":"2024-09-21T11:54:07.440225+0000","last_fullsized":"2024-09-21T11:54:07.440225+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:55:30.061674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439748+0000","last_change":"2024-09-21T11:53:43.398169+0000","last_active":"2024-09-21T11:54:07.439748+0000","last_peered":"2024-09-21T11:54:07.439748+0000","last_clean":"2024-09-21T11:54:07.439748+0000","last_became_active":"2024-09-21T11:53:43.397837+0000","last_became_peered":"2024-09-21T11:53:43.397837+0000","last_unstale":"2024-09-21T11:54:07.439748+0000","last_undegraded":"2024-09-21T11:54:07.439748+0000","last_fullsized":"2024-09-21T11:54:07.439748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:59:51.126413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256440+0000","last_change":"2024-09-21T11:53:39.390677+0000","last_active":"2024-09-21T11:54:07.256440+0000","last_peered":"2024-09-21T11:54:07.256440+0000","last_clean":"2024-09-21T11:54:07.256440+0000","last_became_active":"2024-09-21T11:53:39.388818+0000","last_became_peered":"2024-09-21T11:53:39.388818+0000","last_unstale":"2024-09-21T11:54:07.256440+0000","last_undegraded":"2024-09-21T11:54:07.256440+0000","last_fullsized":"2024-09-21T11:54:07.256440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.706560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255841+0000","last_change":"2024-09-21T11:53:37.387061+0000","last_active":"2024-09-21T11:54:07.255841+0000","last_peered":"2024-09-21T11:54:07.255841+0000","last_clean":"2024-09-21T11:54:07.255841+0000","last_became_active":"2024-09-21T11:53:37.386729+0000","last_became_peered":"2024-09-21T11:53:37.386729+0000","last_unstale":"2024-09-21T11:54:07.255841+0000","last_undegraded":"2024-09-21T11:54:07.255841+0000","last_fullsized":"2024-09-21T11:54:07.255841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:48.070914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440010+0000","last_change":"2024-09-21T11:53:41.398234+0000","last_active":"2024-09-21T11:54:07.440010+0000","last_peered":"2024-09-21T11:54:07.440010+0000","last_clean":"2024-09-21T11:54:07.440010+0000","last_became_active":"2024-09-21T11:53:41.397918+0000","last_became_peered":"2024-09-21T11:53:41.397918+0000","last_unstale":"2024-09-21T11:54:07.440010+0000","last_undegraded":"2024-09-21T11:54:07.440010+0000","last_fullsized":"2024-09-21T11:54:07.440010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:09:13.216152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439863+0000","last_change":"2024-09-21T11:53:43.441108+0000","last_active":"2024-09-21T11:54:07.439863+0000","last_peered":"2024-09-21T11:54:07.439863+0000","last_clean":"2024-09-21T11:54:07.439863+0000","last_became_active":"2024-09-21T11:53:43.440784+0000","last_became_peered":"2024-09-21T11:53:43.440784+0000","last_unstale":"2024-09-21T11:54:07.439863+0000","last_undegraded":"2024-09-21T11:54:07.439863+0000","last_fullsized":"2024-09-21T11:54:07.439863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:51:00.192160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"62'17","reported_seq":54,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256446+0000","last_change":"2024-09-21T11:53:39.386590+0000","last_active":"2024-09-21T11:54:07.256446+0000","last_peered":"2024-09-21T11:54:07.256446+0000","last_clean":"2024-09-21T11:54:07.256446+0000","last_became_active":"2024-09-21T11:53:39.386273+0000","last_became_peered":"2024-09-21T11:53:39.386273+0000","last_unstale":"2024-09-21T11:54:07.256446+0000","last_undegraded":"2024-09-21T11:54:07.256446+0000","last_fullsized":"2024-09-21T11:54:07.256446+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:32.854274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440312+0000","last_change":"2024-09-21T11:53:37.380853+0000","last_active":"2024-09-21T11:54:07.440312+0000","last_peered":"2024-09-21T11:54:07.440312+0000","last_clean":"2024-09-21T11:54:07.440312+0000","last_became_active":"2024-09-21T11:53:37.380546+0000","last_became_peered":"2024-09-21T11:53:37.380546+0000","last_unstale":"2024-09-21T11:54:07.440312+0000","last_undegraded":"2024-09-21T11:54:07.440312+0000","last_fullsized":"2024-09-21T11:54:07.440312+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:22.487427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255862+0000","last_change":"2024-09-21T11:53:41.400213+0000","last_active":"2024-09-21T11:54:07.255862+0000","last_peered":"2024-09-21T11:54:07.255862+0000","last_clean":"2024-09-21T11:54:07.255862+0000","last_became_active":"2024-09-21T11:53:41.399677+0000","last_became_peered":"2024-09-21T11:53:41.399677+0000","last_unstale":"2024-09-21T11:54:07.255862+0000","last_undegraded":"2024-09-21T11:54:07.255862+0000","last_fullsized":"2024-09-21T11:54:07.255862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:35:09.784036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257566+0000","last_change":"2024-09-21T11:53:43.397956+0000","last_active":"2024-09-21T11:54:07.257566+0000","last_peered":"2024-09-21T11:54:07.257566+0000","last_clean":"2024-09-21T11:54:07.257566+0000","last_became_active":"2024-09-21T11:53:43.397604+0000","last_became_peered":"2024-09-21T11:53:43.397604+0000","last_unstale":"2024-09-21T11:54:07.257566+0000","last_undegraded":"2024-09-21T11:54:07.257566+0000","last_fullsized":"2024-09-21T11:54:07.257566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:53.012062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256132+0000","last_change":"2024-09-21T11:53:39.387225+0000","last_active":"2024-09-21T11:54:07.256132+0000","last_peered":"2024-09-21T11:54:07.256132+0000","last_clean":"2024-09-21T11:54:07.256132+0000","last_became_active":"2024-09-21T11:53:39.386059+0000","last_became_peered":"2024-09-21T11:53:39.386059+0000","last_unstale":"2024-09-21T11:54:07.256132+0000","last_undegraded":"2024-09-21T11:54:07.256132+0000","last_fullsized":"2024-09-21T11:54:07.256132+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:32:42.582680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439541+0000","last_change":"2024-09-21T11:53:37.383041+0000","last_active":"2024-09-21T11:54:07.439541+0000","last_peered":"2024-09-21T11:54:07.439541+0000","last_clean":"2024-09-21T11:54:07.439541+0000","last_became_active":"2024-09-21T11:53:37.382747+0000","last_became_peered":"2024-09-21T11:53:37.382747+0000","last_unstale":"2024-09-21T11:54:07.439541+0000","last_undegraded":"2024-09-21T11:54:07.439541+0000","last_fullsized":"2024-09-21T11:54:07.439541+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:09.032346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438430+0000","last_change":"2024-09-21T11:53:41.399244+0000","last_active":"2024-09-21T11:54:12.438430+0000","last_peered":"2024-09-21T11:54:12.438430+0000","last_clean":"2024-09-21T11:54:12.438430+0000","last_became_active":"2024-09-21T11:53:41.398929+0000","last_became_peered":"2024-09-21T11:53:41.398929+0000","last_unstale":"2024-09-21T11:54:12.438430+0000","last_undegraded":"2024-09-21T11:54:12.438430+0000","last_fullsized":"2024-09-21T11:54:12.438430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:52:56.951308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256224+0000","last_change":"2024-09-21T11:53:43.393868+0000","last_active":"2024-09-21T11:54:07.256224+0000","last_peered":"2024-09-21T11:54:07.256224+0000","last_clean":"2024-09-21T11:54:07.256224+0000","last_became_active":"2024-09-21T11:53:43.392748+0000","last_became_peered":"2024-09-21T11:53:43.392748+0000","last_unstale":"2024-09-21T11:54:07.256224+0000","last_undegraded":"2024-09-21T11:54:07.256224+0000","last_fullsized":"2024-09-21T11:54:07.256224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:40:25.325014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441903+0000","last_change":"2024-09-21T11:53:39.386285+0000","last_active":"2024-09-21T11:54:07.441903+0000","last_peered":"2024-09-21T11:54:07.441903+0000","last_clean":"2024-09-21T11:54:07.441903+0000","last_became_active":"2024-09-21T11:53:39.385986+0000","last_became_peered":"2024-09-21T11:53:39.385986+0000","last_unstale":"2024-09-21T11:54:07.441903+0000","last_undegraded":"2024-09-21T11:54:07.441903+0000","last_fullsized":"2024-09-21T11:54:07.441903+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:40.388787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439949+0000","last_change":"2024-09-21T11:53:37.385798+0000","last_active":"2024-09-21T11:54:07.439949+0000","last_peered":"2024-09-21T11:54:07.439949+0000","last_clean":"2024-09-21T11:54:07.439949+0000","last_became_active":"2024-09-21T11:53:37.384414+0000","last_became_peered":"2024-09-21T11:53:37.384414+0000","last_unstale":"2024-09-21T11:54:07.439949+0000","last_undegraded":"2024-09-21T11:54:07.439949+0000","last_fullsized":"2024-09-21T11:54:07.439949+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:29.312339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256633+0000","last_change":"2024-09-21T11:53:41.403835+0000","last_active":"2024-09-21T11:54:07.256633+0000","last_peered":"2024-09-21T11:54:07.256633+0000","last_clean":"2024-09-21T11:54:07.256633+0000","last_became_active":"2024-09-21T11:53:41.403548+0000","last_became_peered":"2024-09-21T11:53:41.403548+0000","last_unstale":"2024-09-21T11:54:07.256633+0000","last_undegraded":"2024-09-21T11:54:07.256633+0000","last_fullsized":"2024-09-21T11:54:07.256633+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:29:57.265766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438869+0000","last_change":"2024-09-21T11:53:43.396145+0000","last_active":"2024-09-21T11:54:07.438869+0000","last_peered":"2024-09-21T11:54:07.438869+0000","last_clean":"2024-09-21T11:54:07.438869+0000","last_became_active":"2024-09-21T11:53:43.395723+0000","last_became_peered":"2024-09-21T11:53:43.395723+0000","last_unstale":"2024-09-21T11:54:07.438869+0000","last_undegraded":"2024-09-21T11:54:07.438869+0000","last_fullsized":"2024-09-21T11:54:07.438869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:45.716959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"62'19","reported_seq":57,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440059+0000","last_change":"2024-09-21T11:53:39.391483+0000","last_active":"2024-09-21T11:54:07.440059+0000","last_peered":"2024-09-21T11:54:07.440059+0000","last_clean":"2024-09-21T11:54:07.440059+0000","last_became_active":"2024-09-21T11:53:39.391172+0000","last_became_peered":"2024-09-21T11:53:39.391172+0000","last_unstale":"2024-09-21T11:54:07.440059+0000","last_undegraded":"2024-09-21T11:54:07.440059+0000","last_fullsized":"2024-09-21T11:54:07.440059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:54.323155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439263+0000","last_change":"2024-09-21T11:53:37.390266+0000","last_active":"2024-09-21T11:54:07.439263+0000","last_peered":"2024-09-21T11:54:07.439263+0000","last_clean":"2024-09-21T11:54:07.439263+0000","last_became_active":"2024-09-21T11:53:37.389929+0000","last_became_peered":"2024-09-21T11:53:37.389929+0000","last_unstale":"2024-09-21T11:54:07.439263+0000","last_undegraded":"2024-09-21T11:54:07.439263+0000","last_fullsized":"2024-09-21T11:54:07.439263+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:27.832760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256875+0000","last_change":"2024-09-21T11:53:41.403299+0000","last_active":"2024-09-21T11:54:07.256875+0000","last_peered":"2024-09-21T11:54:07.256875+0000","last_clean":"2024-09-21T11:54:07.256875+0000","last_became_active":"2024-09-21T11:53:41.403020+0000","last_became_peered":"2024-09-21T11:53:41.403020+0000","last_unstale":"2024-09-21T11:54:07.256875+0000","last_undegraded":"2024-09-21T11:54:07.256875+0000","last_fullsized":"2024-09-21T11:54:07.256875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:51:52.500090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438277+0000","last_change":"2024-09-21T11:53:43.400537+0000","last_active":"2024-09-21T11:54:07.438277+0000","last_peered":"2024-09-21T11:54:07.438277+0000","last_clean":"2024-09-21T11:54:07.438277+0000","last_became_active":"2024-09-21T11:53:43.400271+0000","last_became_peered":"2024-09-21T11:53:43.400271+0000","last_unstale":"2024-09-21T11:54:07.438277+0000","last_undegraded":"2024-09-21T11:54:07.438277+0000","last_fullsized":"2024-09-21T11:54:07.438277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:38.490320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"62'12","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255921+0000","last_change":"2024-09-21T11:53:39.386755+0000","last_active":"2024-09-21T11:54:07.255921+0000","last_peered":"2024-09-21T11:54:07.255921+0000","last_clean":"2024-09-21T11:54:07.255921+0000","last_became_active":"2024-09-21T11:53:39.386455+0000","last_became_peered":"2024-09-21T11:53:39.386455+0000","last_unstale":"2024-09-21T11:54:07.255921+0000","last_undegraded":"2024-09-21T11:54:07.255921+0000","last_fullsized":"2024-09-21T11:54:07.255921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:11.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439913+0000","last_change":"2024-09-21T11:53:37.386547+0000","last_active":"2024-09-21T11:54:07.439913+0000","last_peered":"2024-09-21T11:54:07.439913+0000","last_clean":"2024-09-21T11:54:07.439913+0000","last_became_active":"2024-09-21T11:53:37.386121+0000","last_became_peered":"2024-09-21T11:53:37.386121+0000","last_unstale":"2024-09-21T11:54:07.439913+0000","last_undegraded":"2024-09-21T11:54:07.439913+0000","last_fullsized":"2024-09-21T11:54:07.439913+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:27.369893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256029+0000","last_change":"2024-09-21T11:53:41.402854+0000","last_active":"2024-09-21T11:54:07.256029+0000","last_peered":"2024-09-21T11:54:07.256029+0000","last_clean":"2024-09-21T11:54:07.256029+0000","last_became_active":"2024-09-21T11:53:41.402558+0000","last_became_peered":"2024-09-21T11:53:41.402558+0000","last_unstale":"2024-09-21T11:54:07.256029+0000","last_undegraded":"2024-09-21T11:54:07.256029+0000","last_fullsized":"2024-09-21T11:54:07.256029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:16.438507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438519+0000","last_change":"2024-09-21T11:53:43.402262+0000","last_active":"2024-09-21T11:54:07.438519+0000","last_peered":"2024-09-21T11:54:07.438519+0000","last_clean":"2024-09-21T11:54:07.438519+0000","last_became_active":"2024-09-21T11:53:43.401930+0000","last_became_peered":"2024-09-21T11:53:43.401930+0000","last_unstale":"2024-09-21T11:54:07.438519+0000","last_undegraded":"2024-09-21T11:54:07.438519+0000","last_fullsized":"2024-09-21T11:54:07.438519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:16.473753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"62'15","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439141+0000","last_change":"2024-09-21T11:53:39.384015+0000","last_active":"2024-09-21T11:54:07.439141+0000","last_peered":"2024-09-21T11:54:07.439141+0000","last_clean":"2024-09-21T11:54:07.439141+0000","last_became_active":"2024-09-21T11:53:39.383578+0000","last_became_peered":"2024-09-21T11:53:39.383578+0000","last_unstale":"2024-09-21T11:54:07.439141+0000","last_undegraded":"2024-09-21T11:54:07.439141+0000","last_fullsized":"2024-09-21T11:54:07.439141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:16.228287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"55'2","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439293+0000","last_change":"2024-09-21T11:53:37.387440+0000","last_active":"2024-09-21T11:54:07.439293+0000","last_peered":"2024-09-21T11:54:07.439293+0000","last_clean":"2024-09-21T11:54:07.439293+0000","last_became_active":"2024-09-21T11:53:37.387070+0000","last_became_peered":"2024-09-21T11:53:37.387070+0000","last_unstale":"2024-09-21T11:54:07.439293+0000","last_undegraded":"2024-09-21T11:54:07.439293+0000","last_fullsized":"2024-09-21T11:54:07.439293+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:42.938252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438808+0000","last_change":"2024-09-21T11:53:41.398931+0000","last_active":"2024-09-21T11:54:12.438808+0000","last_peered":"2024-09-21T11:54:12.438808+0000","last_clean":"2024-09-21T11:54:12.438808+0000","last_became_active":"2024-09-21T11:53:41.398656+0000","last_became_peered":"2024-09-21T11:53:41.398656+0000","last_unstale":"2024-09-21T11:54:12.438808+0000","last_undegraded":"2024-09-21T11:54:12.438808+0000","last_fullsized":"2024-09-21T11:54:12.438808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:28.845397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438706+0000","last_change":"2024-09-21T11:53:43.441111+0000","last_active":"2024-09-21T11:54:07.438706+0000","last_peered":"2024-09-21T11:54:07.438706+0000","last_clean":"2024-09-21T11:54:07.438706+0000","last_became_active":"2024-09-21T11:53:43.440821+0000","last_became_peered":"2024-09-21T11:53:43.440821+0000","last_unstale":"2024-09-21T11:54:07.438706+0000","last_undegraded":"2024-09-21T11:54:07.438706+0000","last_fullsized":"2024-09-21T11:54:07.438706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:24.397724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440865+0000","last_change":"2024-09-21T11:53:37.384903+0000","last_active":"2024-09-21T11:54:07.440865+0000","last_peered":"2024-09-21T11:54:07.440865+0000","last_clean":"2024-09-21T11:54:07.440865+0000","last_became_active":"2024-09-21T11:53:37.384592+0000","last_became_peered":"2024-09-21T11:53:37.384592+0000","last_unstale":"2024-09-21T11:54:07.440865+0000","last_undegraded":"2024-09-21T11:54:07.440865+0000","last_fullsized":"2024-09-21T11:54:07.440865+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:26.693796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"62'6","reported_seq":35,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440701+0000","last_change":"2024-09-21T11:53:39.390279+0000","last_active":"2024-09-21T11:54:07.440701+0000","last_peered":"2024-09-21T11:54:07.440701+0000","last_clean":"2024-09-21T11:54:07.440701+0000","last_became_active":"2024-09-21T11:53:39.389878+0000","last_became_peered":"2024-09-21T11:53:39.389878+0000","last_unstale":"2024-09-21T11:54:07.440701+0000","last_undegraded":"2024-09-21T11:54:07.440701+0000","last_fullsized":"2024-09-21T11:54:07.440701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:13:07.725648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439573+0000","last_change":"2024-09-21T11:53:41.392302+0000","last_active":"2024-09-21T11:54:07.439573+0000","last_peered":"2024-09-21T11:54:07.439573+0000","last_clean":"2024-09-21T11:54:07.439573+0000","last_became_active":"2024-09-21T11:53:41.392028+0000","last_became_peered":"2024-09-21T11:53:41.392028+0000","last_unstale":"2024-09-21T11:54:07.439573+0000","last_undegraded":"2024-09-21T11:54:07.439573+0000","last_fullsized":"2024-09-21T11:54:07.439573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:34:43.601518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438514+0000","last_change":"2024-09-21T11:53:43.443026+0000","last_active":"2024-09-21T11:54:07.438514+0000","last_peered":"2024-09-21T11:54:07.438514+0000","last_clean":"2024-09-21T11:54:07.438514+0000","last_became_active":"2024-09-21T11:53:43.442481+0000","last_became_peered":"2024-09-21T11:53:43.442481+0000","last_unstale":"2024-09-21T11:54:07.438514+0000","last_undegraded":"2024-09-21T11:54:07.438514+0000","last_fullsized":"2024-09-21T11:54:07.438514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:28:44.477080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441560+0000","last_change":"2024-09-21T11:53:39.384630+0000","last_active":"2024-09-21T11:54:07.441560+0000","last_peered":"2024-09-21T11:54:07.441560+0000","last_clean":"2024-09-21T11:54:07.441560+0000","last_became_active":"2024-09-21T11:53:39.384041+0000","last_became_peered":"2024-09-21T11:53:39.384041+0000","last_unstale":"2024-09-21T11:54:07.441560+0000","last_undegraded":"2024-09-21T11:54:07.441560+0000","last_fullsized":"2024-09-21T11:54:07.441560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:06.761186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439922+0000","last_change":"2024-09-21T11:53:37.384725+0000","last_active":"2024-09-21T11:54:07.439922+0000","last_peered":"2024-09-21T11:54:07.439922+0000","last_clean":"2024-09-21T11:54:07.439922+0000","last_became_active":"2024-09-21T11:53:37.384415+0000","last_became_peered":"2024-09-21T11:53:37.384415+0000","last_unstale":"2024-09-21T11:54:07.439922+0000","last_undegraded":"2024-09-21T11:54:07.439922+0000","last_fullsized":"2024-09-21T11:54:07.439922+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:30.878201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440610+0000","last_change":"2024-09-21T11:53:41.392596+0000","last_active":"2024-09-21T11:54:07.440610+0000","last_peered":"2024-09-21T11:54:07.440610+0000","last_clean":"2024-09-21T11:54:07.440610+0000","last_became_active":"2024-09-21T11:53:41.392260+0000","last_became_peered":"2024-09-21T11:53:41.392260+0000","last_unstale":"2024-09-21T11:54:07.440610+0000","last_undegraded":"2024-09-21T11:54:07.440610+0000","last_fullsized":"2024-09-21T11:54:07.440610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:14.028758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.257369+0000","last_change":"2024-09-21T11:53:43.396305+0000","last_active":"2024-09-21T11:54:07.257369+0000","last_peered":"2024-09-21T11:54:07.257369+0000","last_clean":"2024-09-21T11:54:07.257369+0000","last_became_active":"2024-09-21T11:53:43.396007+0000","last_became_peered":"2024-09-21T11:53:43.396007+0000","last_unstale":"2024-09-21T11:54:07.257369+0000","last_undegraded":"2024-09-21T11:54:07.257369+0000","last_fullsized":"2024-09-21T11:54:07.257369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:49.615330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440493+0000","last_change":"2024-09-21T11:53:39.383700+0000","last_active":"2024-09-21T11:54:07.440493+0000","last_peered":"2024-09-21T11:54:07.440493+0000","last_clean":"2024-09-21T11:54:07.440493+0000","last_became_active":"2024-09-21T11:53:39.383340+0000","last_became_peered":"2024-09-21T11:53:39.383340+0000","last_unstale":"2024-09-21T11:54:07.440493+0000","last_undegraded":"2024-09-21T11:54:07.440493+0000","last_fullsized":"2024-09-21T11:54:07.440493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:06.692192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439506+0000","last_change":"2024-09-21T11:53:37.379487+0000","last_active":"2024-09-21T11:54:07.439506+0000","last_peered":"2024-09-21T11:54:07.439506+0000","last_clean":"2024-09-21T11:54:07.439506+0000","last_became_active":"2024-09-21T11:53:37.379165+0000","last_became_peered":"2024-09-21T11:53:37.379165+0000","last_unstale":"2024-09-21T11:54:07.439506+0000","last_undegraded":"2024-09-21T11:54:07.439506+0000","last_fullsized":"2024-09-21T11:54:07.439506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:52.614566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"62'8","reported_seq":39,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438723+0000","last_change":"2024-09-21T11:53:41.396552+0000","last_active":"2024-09-21T11:54:12.438723+0000","last_peered":"2024-09-21T11:54:12.438723+0000","last_clean":"2024-09-21T11:54:12.438723+0000","last_became_active":"2024-09-21T11:53:41.396215+0000","last_became_peered":"2024-09-21T11:53:41.396215+0000","last_unstale":"2024-09-21T11:54:12.438723+0000","last_undegraded":"2024-09-21T11:54:12.438723+0000","last_fullsized":"2024-09-21T11:54:12.438723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:17.331236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256659+0000","last_change":"2024-09-21T11:53:43.395899+0000","last_active":"2024-09-21T11:54:07.256659+0000","last_peered":"2024-09-21T11:54:07.256659+0000","last_clean":"2024-09-21T11:54:07.256659+0000","last_became_active":"2024-09-21T11:53:43.395607+0000","last_became_peered":"2024-09-21T11:53:43.395607+0000","last_unstale":"2024-09-21T11:54:07.256659+0000","last_undegraded":"2024-09-21T11:54:07.256659+0000","last_fullsized":"2024-09-21T11:54:07.256659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:52.180882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"62'10","reported_seq":41,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440141+0000","last_change":"2024-09-21T11:53:39.383371+0000","last_active":"2024-09-21T11:54:07.440141+0000","last_peered":"2024-09-21T11:54:07.440141+0000","last_clean":"2024-09-21T11:54:07.440141+0000","last_became_active":"2024-09-21T11:53:39.383062+0000","last_became_peered":"2024-09-21T11:53:39.383062+0000","last_unstale":"2024-09-21T11:54:07.440141+0000","last_undegraded":"2024-09-21T11:54:07.440141+0000","last_fullsized":"2024-09-21T11:54:07.440141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:52.294339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439830+0000","last_change":"2024-09-21T11:53:37.387643+0000","last_active":"2024-09-21T11:54:07.439830+0000","last_peered":"2024-09-21T11:54:07.439830+0000","last_clean":"2024-09-21T11:54:07.439830+0000","last_became_active":"2024-09-21T11:53:37.387318+0000","last_became_peered":"2024-09-21T11:53:37.387318+0000","last_unstale":"2024-09-21T11:54:07.439830+0000","last_undegraded":"2024-09-21T11:54:07.439830+0000","last_fullsized":"2024-09-21T11:54:07.439830+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:41.697395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"62'8","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:12.438796+0000","last_change":"2024-09-21T11:53:41.401581+0000","last_active":"2024-09-21T11:54:12.438796+0000","last_peered":"2024-09-21T11:54:12.438796+0000","last_clean":"2024-09-21T11:54:12.438796+0000","last_became_active":"2024-09-21T11:53:41.401284+0000","last_became_peered":"2024-09-21T11:53:41.401284+0000","last_unstale":"2024-09-21T11:54:12.438796+0000","last_undegraded":"2024-09-21T11:54:12.438796+0000","last_fullsized":"2024-09-21T11:54:12.438796+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:11.901950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441160+0000","last_change":"2024-09-21T11:53:43.396255+0000","last_active":"2024-09-21T11:54:07.441160+0000","last_peered":"2024-09-21T11:54:07.441160+0000","last_clean":"2024-09-21T11:54:07.441160+0000","last_became_active":"2024-09-21T11:53:43.395938+0000","last_became_peered":"2024-09-21T11:53:43.395938+0000","last_unstale":"2024-09-21T11:54:07.441160+0000","last_undegraded":"2024-09-21T11:54:07.441160+0000","last_fullsized":"2024-09-21T11:54:07.441160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:42.292829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255946+0000","last_change":"2024-09-21T11:53:39.390832+0000","last_active":"2024-09-21T11:54:07.255946+0000","last_peered":"2024-09-21T11:54:07.255946+0000","last_clean":"2024-09-21T11:54:07.255946+0000","last_became_active":"2024-09-21T11:53:39.388530+0000","last_became_peered":"2024-09-21T11:53:39.388530+0000","last_unstale":"2024-09-21T11:54:07.255946+0000","last_undegraded":"2024-09-21T11:54:07.255946+0000","last_fullsized":"2024-09-21T11:54:07.255946+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:00.056398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256344+0000","last_change":"2024-09-21T11:53:37.383270+0000","last_active":"2024-09-21T11:54:07.256344+0000","last_peered":"2024-09-21T11:54:07.256344+0000","last_clean":"2024-09-21T11:54:07.256344+0000","last_became_active":"2024-09-21T11:53:37.382918+0000","last_became_peered":"2024-09-21T11:53:37.382918+0000","last_unstale":"2024-09-21T11:54:07.256344+0000","last_undegraded":"2024-09-21T11:54:07.256344+0000","last_fullsized":"2024-09-21T11:54:07.256344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:50.086823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.255727+0000","last_change":"2024-09-21T11:53:41.390687+0000","last_active":"2024-09-21T11:54:07.255727+0000","last_peered":"2024-09-21T11:54:07.255727+0000","last_clean":"2024-09-21T11:54:07.255727+0000","last_became_active":"2024-09-21T11:53:41.390333+0000","last_became_peered":"2024-09-21T11:53:41.390333+0000","last_unstale":"2024-09-21T11:54:07.255727+0000","last_undegraded":"2024-09-21T11:54:07.255727+0000","last_fullsized":"2024-09-21T11:54:07.255727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:42.377380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439111+0000","last_change":"2024-09-21T11:53:43.399132+0000","last_active":"2024-09-21T11:54:07.439111+0000","last_peered":"2024-09-21T11:54:07.439111+0000","last_clean":"2024-09-21T11:54:07.439111+0000","last_became_active":"2024-09-21T11:53:43.398846+0000","last_became_peered":"2024-09-21T11:53:43.398846+0000","last_unstale":"2024-09-21T11:54:07.439111+0000","last_undegraded":"2024-09-21T11:54:07.439111+0000","last_fullsized":"2024-09-21T11:54:07.439111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:42.420048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"62'9","reported_seq":42,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256053+0000","last_change":"2024-09-21T11:53:39.387293+0000","last_active":"2024-09-21T11:54:07.256053+0000","last_peered":"2024-09-21T11:54:07.256053+0000","last_clean":"2024-09-21T11:54:07.256053+0000","last_became_active":"2024-09-21T11:53:39.386083+0000","last_became_peered":"2024-09-21T11:53:39.386083+0000","last_unstale":"2024-09-21T11:54:07.256053+0000","last_undegraded":"2024-09-21T11:54:07.256053+0000","last_fullsized":"2024-09-21T11:54:07.256053+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:17.987422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439118+0000","last_change":"2024-09-21T11:53:37.387767+0000","last_active":"2024-09-21T11:54:07.439118+0000","last_peered":"2024-09-21T11:54:07.439118+0000","last_clean":"2024-09-21T11:54:07.439118+0000","last_became_active":"2024-09-21T11:53:37.387426+0000","last_became_peered":"2024-09-21T11:53:37.387426+0000","last_unstale":"2024-09-21T11:54:07.439118+0000","last_undegraded":"2024-09-21T11:54:07.439118+0000","last_fullsized":"2024-09-21T11:54:07.439118+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:49.280376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439387+0000","last_change":"2024-09-21T11:53:41.392884+0000","last_active":"2024-09-21T11:54:07.439387+0000","last_peered":"2024-09-21T11:54:07.439387+0000","last_clean":"2024-09-21T11:54:07.439387+0000","last_became_active":"2024-09-21T11:53:41.392588+0000","last_became_peered":"2024-09-21T11:53:41.392588+0000","last_unstale":"2024-09-21T11:54:07.439387+0000","last_undegraded":"2024-09-21T11:54:07.439387+0000","last_fullsized":"2024-09-21T11:54:07.439387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:07.692056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440633+0000","last_change":"2024-09-21T11:53:43.399754+0000","last_active":"2024-09-21T11:54:07.440633+0000","last_peered":"2024-09-21T11:54:07.440633+0000","last_clean":"2024-09-21T11:54:07.440633+0000","last_became_active":"2024-09-21T11:53:43.399473+0000","last_became_peered":"2024-09-21T11:53:43.399473+0000","last_unstale":"2024-09-21T11:54:07.440633+0000","last_undegraded":"2024-09-21T11:54:07.440633+0000","last_fullsized":"2024-09-21T11:54:07.440633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:39.435159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439231+0000","last_change":"2024-09-21T11:53:39.387978+0000","last_active":"2024-09-21T11:54:07.439231+0000","last_peered":"2024-09-21T11:54:07.439231+0000","last_clean":"2024-09-21T11:54:07.439231+0000","last_became_active":"2024-09-21T11:53:39.387490+0000","last_became_peered":"2024-09-21T11:53:39.387490+0000","last_unstale":"2024-09-21T11:54:07.439231+0000","last_undegraded":"2024-09-21T11:54:07.439231+0000","last_fullsized":"2024-09-21T11:54:07.439231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:36.333886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439651+0000","last_change":"2024-09-21T11:53:37.386371+0000","last_active":"2024-09-21T11:54:07.439651+0000","last_peered":"2024-09-21T11:54:07.439651+0000","last_clean":"2024-09-21T11:54:07.439651+0000","last_became_active":"2024-09-21T11:53:37.385465+0000","last_became_peered":"2024-09-21T11:53:37.385465+0000","last_unstale":"2024-09-21T11:54:07.439651+0000","last_undegraded":"2024-09-21T11:54:07.439651+0000","last_fullsized":"2024-09-21T11:54:07.439651+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:39:07.498543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438402+0000","last_change":"2024-09-21T11:53:41.392423+0000","last_active":"2024-09-21T11:54:07.438402+0000","last_peered":"2024-09-21T11:54:07.438402+0000","last_clean":"2024-09-21T11:54:07.438402+0000","last_became_active":"2024-09-21T11:53:41.392152+0000","last_became_peered":"2024-09-21T11:53:41.392152+0000","last_unstale":"2024-09-21T11:54:07.438402+0000","last_undegraded":"2024-09-21T11:54:07.438402+0000","last_fullsized":"2024-09-21T11:54:07.438402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:08.411208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439711+0000","last_change":"2024-09-21T11:53:43.441817+0000","last_active":"2024-09-21T11:54:07.439711+0000","last_peered":"2024-09-21T11:54:07.439711+0000","last_clean":"2024-09-21T11:54:07.439711+0000","last_became_active":"2024-09-21T11:53:43.441520+0000","last_became_peered":"2024-09-21T11:53:43.441520+0000","last_unstale":"2024-09-21T11:54:07.439711+0000","last_undegraded":"2024-09-21T11:54:07.439711+0000","last_fullsized":"2024-09-21T11:54:07.439711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:52.951050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"62'4","reported_seq":32,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438751+0000","last_change":"2024-09-21T11:53:39.386728+0000","last_active":"2024-09-21T11:54:07.438751+0000","last_peered":"2024-09-21T11:54:07.438751+0000","last_clean":"2024-09-21T11:54:07.438751+0000","last_became_active":"2024-09-21T11:53:39.386460+0000","last_became_peered":"2024-09-21T11:53:39.386460+0000","last_unstale":"2024-09-21T11:54:07.438751+0000","last_undegraded":"2024-09-21T11:54:07.438751+0000","last_fullsized":"2024-09-21T11:54:07.438751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:26.881694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.440301+0000","last_change":"2024-09-21T11:53:37.390304+0000","last_active":"2024-09-21T11:54:07.440301+0000","last_peered":"2024-09-21T11:54:07.440301+0000","last_clean":"2024-09-21T11:54:07.440301+0000","last_became_active":"2024-09-21T11:53:37.389841+0000","last_became_peered":"2024-09-21T11:53:37.389841+0000","last_unstale":"2024-09-21T11:54:07.440301+0000","last_undegraded":"2024-09-21T11:54:07.440301+0000","last_fullsized":"2024-09-21T11:54:07.440301+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:42.531237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438751+0000","last_change":"2024-09-21T11:53:41.397901+0000","last_active":"2024-09-21T11:54:07.438751+0000","last_peered":"2024-09-21T11:54:07.438751+0000","last_clean":"2024-09-21T11:54:07.438751+0000","last_became_active":"2024-09-21T11:53:41.396082+0000","last_became_peered":"2024-09-21T11:53:41.396082+0000","last_unstale":"2024-09-21T11:54:07.438751+0000","last_undegraded":"2024-09-21T11:54:07.438751+0000","last_fullsized":"2024-09-21T11:54:07.438751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:35:54.054554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438517+0000","last_change":"2024-09-21T11:53:43.394884+0000","last_active":"2024-09-21T11:54:07.438517+0000","last_peered":"2024-09-21T11:54:07.438517+0000","last_clean":"2024-09-21T11:54:07.438517+0000","last_became_active":"2024-09-21T11:53:43.394563+0000","last_became_peered":"2024-09-21T11:53:43.394563+0000","last_unstale":"2024-09-21T11:54:07.438517+0000","last_undegraded":"2024-09-21T11:54:07.438517+0000","last_fullsized":"2024-09-21T11:54:07.438517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:17:05.190359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.256196+0000","last_change":"2024-09-21T11:53:43.398064+0000","last_active":"2024-09-21T11:54:07.256196+0000","last_peered":"2024-09-21T11:54:07.256196+0000","last_clean":"2024-09-21T11:54:07.256196+0000","last_became_active":"2024-09-21T11:53:43.397660+0000","last_became_peered":"2024-09-21T11:53:43.397660+0000","last_unstale":"2024-09-21T11:54:07.256196+0000","last_undegraded":"2024-09-21T11:54:07.256196+0000","last_fullsized":"2024-09-21T11:54:07.256196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:42.378994+0000","last_clean_scrub_stamp":"2024-09-21T11:53:42.378994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:48.365575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.438923+0000","last_change":"2024-09-21T11:53:37.378670+0000","last_active":"2024-09-21T11:54:07.438923+0000","last_peered":"2024-09-21T11:54:07.438923+0000","last_clean":"2024-09-21T11:54:07.438923+0000","last_became_active":"2024-09-21T11:53:37.378426+0000","last_became_peered":"2024-09-21T11:53:37.378426+0000","last_unstale":"2024-09-21T11:54:07.438923+0000","last_undegraded":"2024-09-21T11:54:07.438923+0000","last_fullsized":"2024-09-21T11:54:07.438923+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:36.359009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:36.359009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:16:23.530374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"62'11","reported_seq":45,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.439481+0000","last_change":"2024-09-21T11:53:39.394468+0000","last_active":"2024-09-21T11:54:07.439481+0000","last_peered":"2024-09-21T11:54:07.439481+0000","last_clean":"2024-09-21T11:54:07.439481+0000","last_became_active":"2024-09-21T11:53:39.394153+0000","last_became_peered":"2024-09-21T11:53:39.394153+0000","last_unstale":"2024-09-21T11:54:07.439481+0000","last_undegraded":"2024-09-21T11:54:07.439481+0000","last_fullsized":"2024-09-21T11:54:07.439481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:38.366463+0000","last_clean_scrub_stamp":"2024-09-21T11:53:38.366463+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:23.957950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-21T11:54:07.441321+0000","last_change":"2024-09-21T11:53:41.395419+0000","last_active":"2024-09-21T11:54:07.441321+0000","last_peered":"2024-09-21T11:54:07.441321+0000","last_clean":"2024-09-21T11:54:07.441321+0000","last_became_active":"2024-09-21T11:53:41.395090+0000","last_became_peered":"2024-09-21T11:53:41.395090+0000","last_unstale":"2024-09-21T11:54:07.441321+0000","last_undegraded":"2024-09-21T11:54:07.441321+0000","last_fullsized":"2024-09-21T11:54:07.441321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:53:40.370968+0000","last_clean_scrub_stamp":"2024-09-21T11:53:40.370968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:50:38.958782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":50,"seq":214748364811,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712688,"statfs":{"total":95995035648,"available":95961792512,"internally_reserved":0,"allocated":1310720,"data_stored":882909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593746,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712712,"statfs":{"total":95995035648,"available":95961817088,"internally_reserved":0,"allocated":1286144,"data_stored":881388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789975,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717444,"statfs":{"total":95995035648,"available":95966662656,"internally_reserved":0,"allocated":696320,"data_stored":291379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986206,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31896,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713256,"statfs":{"total":95995035648,"available":95962374144,"internally_reserved":0,"allocated":724992,"data_stored":292618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149731,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31912,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713240,"statfs":{"total":95995035648,"available":95962357760,"internally_reserved":0,"allocated":741376,"data_stored":293353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717448,"statfs":{"total":95995035648,"available":95966666752,"internally_reserved":0,"allocated":692224,"data_stored":291181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":757760,"data_stored":292805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1323008,"data_stored":883624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:54:24.265 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T11:54:24.265 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T11:54:24.265 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T11:54:24.266 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 1795c04c-780f-11ef-baf6-efdc52797490 -- ceph health --format=json 2024-09-21T11:54:24.511 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:25.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 -- 172.21.15.107:0/1368577220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc101c70 msgr2=0x7f83cc102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:25.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1368577220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc101c70 0x7f83cc102050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009860 tx=0x7f83bc02a340 comp rx=0 tx=0).stop 2024-09-21T11:54:25.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 -- 172.21.15.107:0/1368577220 shutdown_connections 2024-09-21T11:54:25.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1368577220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 0x7f83cc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1368577220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc107680 0x7f83cc107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1368577220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc101c70 0x7f83cc102050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 -- 172.21.15.107:0/1368577220 >> 172.21.15.107:0/1368577220 conn(0x7f83cc0fd700 msgr2=0x7f83cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 -- 172.21.15.107:0/1368577220 shutdown_connections 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.108+0000 7f83d0af2640 1 -- 172.21.15.107:0/1368577220 wait complete. 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 Processor -- start 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 -- start start 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc101c70 0x7f83cc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 0x7f83cc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 0x7f83cc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc111190 con 0x7f83cc108000 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc111010 con 0x7f83cc101c70 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83d0af2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f83cc111310 con 0x7f83cc107680 2024-09-21T11:54:25.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 0x7f83cc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83cad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 0x7f83cc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 0x7f83cc199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:43464/0 (socket says 172.21.15.107:43464) 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 -- 172.21.15.107:0/1048021679 learned_addr learned my addr 172.21.15.107:0/1048021679 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83ca59c640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc101c70 0x7f83cc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 -- 172.21.15.107:0/1048021679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc101c70 msgr2=0x7f83cc1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc101c70 0x7f83cc1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 -- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 msgr2=0x7f83cc19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 0x7f83cc19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83c9d9b640 1 -- 172.21.15.107:0/1048021679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1a49c0 con 0x7f83cc107680 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83cad9d640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 0x7f83cc19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.109+0000 7f83ca59c640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc101c70 0x7f83cc1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T11:54:25.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.110+0000 7f83c9d9b640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 0x7f83cc199b00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c970 tx=0x7f83c000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.110+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0007bb0 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.110+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1a4cb0 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.110+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f83c0007d50 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.110+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0004c90 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.110+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a51a0 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.111+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc102050 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.111+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f83c0004eb0 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.112+0000 7f83bb7fe640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83b407b150 0x7f83b407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.112+0000 7f83ca59c640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83b407b150 0x7f83b407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.112+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 7421+0+0 (secure 0 0 0) 0x7f83c009e4c0 con 0x7f83cc107680 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.112+0000 7f83ca59c640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83b407b150 0x7f83b407d610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f83bc000c00 tx=0x7f83bc0058b0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T11:54:25.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.116+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83c0014030 con 0x7f83cc107680 2024-09-21T11:54:25.200 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:24 smithi107 ceph-mon[30293]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:25.200 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:24 smithi107 ceph-mon[30293]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T11:54:25.200 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:24 smithi107 ceph-mon[41097]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:25.200 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:24 smithi107 ceph-mon[41097]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T11:54:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:24 smithi193 ceph-mon[33333]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:24 smithi193 ceph-mon[33333]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-21T11:54:25.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.359+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f83cc109de0 con 0x7f83cc107680 2024-09-21T11:54:25.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.360+0000 7f83bb7fe640 1 -- 172.21.15.107:0/1048021679 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f83c00695d0 con 0x7f83cc107680 2024-09-21T11:54:25.362 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T11:54:25.362 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T11:54:25.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83b407b150 msgr2=0x7f83b407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:25.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83b407b150 0x7f83b407d610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f83bc000c00 tx=0x7f83bc0058b0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 msgr2=0x7f83cc199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T11:54:25.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 0x7f83cc199b00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c970 tx=0x7f83c000ce30 comp rx=0 tx=0).stop 2024-09-21T11:54:25.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 shutdown_connections 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83b407b150 0x7f83b407d610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83cc108000 0x7f83cc19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83cc107680 0x7f83cc199b00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 --2- 172.21.15.107:0/1048021679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83cc101c70 0x7f83cc1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 >> 172.21.15.107:0/1048021679 conn(0x7f83cc0fd700 msgr2=0x7f83cc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 shutdown_connections 2024-09-21T11:54:25.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T11:54:25.362+0000 7f83d0af2640 1 -- 172.21.15.107:0/1048021679 wait complete. 2024-09-21T11:54:26.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1048021679' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T11:54:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1048021679' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T11:54:26.295 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T11:54:26.295 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T11:54:26.295 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T11:54:26.308 INFO:tasks.workunit:Pulling workunits from ref 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:54:26.308 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T11:54:26.308 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T11:54:26.309 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1048021679' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T11:54:26.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T11:54:26.325 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-21T11:54:26.380 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-21T11:54:26.380 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-21T11:54:26.436 INFO:tasks.workunit:timeout=3h 2024-09-21T11:54:26.436 INFO:tasks.workunit:cleanup=True 2024-09-21T11:54:26.436 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:54:26.491 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T11:54:27.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:26 smithi107 ceph-mon[30293]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T11:54:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:26 smithi107 ceph-mon[41097]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T11:54:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:26 smithi193 ceph-mon[33333]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-21T11:54:28.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:54:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:54:29.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:28 smithi107 ceph-mon[30293]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:29.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:28 smithi107 ceph-mon[41097]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:28 smithi193 ceph-mon[33333]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:30.201 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:30.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:31.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:30 smithi193 ceph-mon[33333]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:30 smithi107 ceph-mon[30293]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:30 smithi107 ceph-mon[41097]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:31.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 systemd[1]: Starting Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:54:31.575 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 podman[75577]: 2024-09-21 11:54:31.178004767 +0000 UTC m=+0.015734795 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-21T11:54:31.576 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 podman[75577]: 2024-09-21 11:54:31.291833276 +0000 UTC m=+0.129563341 volume create ba4195b000b9130d9db765dccbaf8e4f3edf322e8d18ea2fe41c5323b87423aa 2024-09-21T11:54:31.576 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 podman[75577]: 2024-09-21 11:54:31.375456529 +0000 UTC m=+0.213186551 container create 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 podman[75577]: 2024-09-21 11:54:31.686906372 +0000 UTC m=+0.524636401 container init 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 podman[75577]: 2024-09-21 11:54:31.692071444 +0000 UTC m=+0.529801471 container start 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.702Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.703Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.703Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.107 port=9094 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.704Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.725Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.725Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:54:32.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.727Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-21T11:54:32.005 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:31.727Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-21T11:54:32.005 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 bash[75577]: 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 2024-09-21T11:54:32.005 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:31 smithi107 systemd[1]: Started Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:54:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:54:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:32.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:32] ENGINE Bus STOPPING 2024-09-21T11:54:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:32] ENGINE Bus STOPPED 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:32] ENGINE Bus STARTING 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:32] ENGINE Serving on http://:::9283 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:54:32] ENGINE Bus STARTED 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:32 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-21T11:54:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:32 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:54:34.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T11:54:34.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-mon[30293]: Deploying daemon grafana.a on smithi193 2024-09-21T11:54:34.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:54:34.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:33.704Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000262436s 2024-09-21T11:54:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T11:54:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-mon[41097]: Deploying daemon grafana.a on smithi193 2024-09-21T11:54:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:33 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:54:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:33 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T11:54:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:33 smithi193 ceph-mon[33333]: Deploying daemon grafana.a on smithi193 2024-09-21T11:54:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:33 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:54:35.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:34 smithi107 ceph-mon[30293]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:34 smithi107 ceph-mon[41097]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:34 smithi193 ceph-mon[33333]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:37.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:36 smithi107 ceph-mon[30293]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:36 smithi107 ceph-mon[41097]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:36 smithi193 ceph-mon[33333]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:38.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:37 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:38.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:37 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:37 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:38.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:54:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:54:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:38 smithi107 ceph-mon[30293]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:38 smithi107 ceph-mon[41097]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:38 smithi193 ceph-mon[33333]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:40.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:40.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:40.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:41.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:40 smithi193 ceph-mon[33333]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:40 smithi107 ceph-mon[30293]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:40 smithi107 ceph-mon[41097]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:42.004 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:41 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:41.707Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002952568s 2024-09-21T11:54:42.690 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:42 smithi193 podman[63230]: 2024-09-21 11:54:42.566814455 +0000 UTC m=+0.152755491 container create 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 (image=quay.io/ceph/grafana:10.4.8, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T11:54:42.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:54:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:43.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:42 smithi193 ceph-mon[33333]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:43.120 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 podman[63230]: 2024-09-21 11:54:43.113501284 +0000 UTC m=+0.699442326 container init 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 (image=quay.io/ceph/grafana:10.4.8, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T11:54:43.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:42 smithi107 ceph-mon[41097]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:42 smithi107 ceph-mon[30293]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:43.372 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 podman[63230]: 2024-09-21 11:54:43.119029574 +0000 UTC m=+0.704970610 container start 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 (image=quay.io/ceph/grafana:10.4.8, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T11:54:43.372 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 bash[63230]: 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 systemd[1]: Started Ceph grafana.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.241396209Z level=info msg="Starting Grafana" version=10.4.8 commit=8c384b71bde142a25ddc01ed5a183ca572b2b7f7 branch=v10.4.x compiled=2024-09-21T11:54:43Z 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242529763Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242545752Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242551316Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242561413Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242566126Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242570753Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242575331Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242580374Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242585023Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242589827Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-21T11:54:43.373 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242598176Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242603314Z level=info msg=Target target=[all] 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242615985Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.24262072Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242625142Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242629744Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242638386Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=settings t=2024-09-21T11:54:43.242643341Z level=info msg="App mode production" 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore t=2024-09-21T11:54:43.243261135Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore t=2024-09-21T11:54:43.243283797Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.244628984Z level=info msg="Starting DB migrations" 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.24553032Z level=info msg="Executing migration" id="create migration_log table" 2024-09-21T11:54:43.374 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.246320488Z level=info msg="Migration successfully executed" id="create migration_log table" duration=789.868µs 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.2467754Z level=info msg="Executing migration" id="create user table" 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.247309726Z level=info msg="Migration successfully executed" id="create user table" duration=534.186µs 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.247744585Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.248250948Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=506.443µs 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.248644993Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.249129612Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=485.064µs 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.249500793Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.249962879Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=460.986µs 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.250947001Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.251386518Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=439.413µs 2024-09-21T11:54:43.375 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.251720941Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-21T11:54:43.376 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.25328169Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.559027ms 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.256171166Z level=info msg="Executing migration" id="create user table v2" 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.25672309Z level=info msg="Migration successfully executed" id="create user table v2" duration=551.743µs 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.257078563Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.257525507Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=445.115µs 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.257855128Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.258295063Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=439.654µs 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.258703666Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.259016798Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=313.172µs 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.259352719Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.259721174Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=367.928µs 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.26007839Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.260839869Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=761.084µs 2024-09-21T11:54:43.377 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.261194013Z level=info msg="Executing migration" id="Update user table charset" 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.26121072Z level=info msg="Migration successfully executed" id="Update user table charset" duration=16.699µs 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.261601947Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.262317757Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=715.438µs 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.264851203Z level=info msg="Executing migration" id="Add missing user data" 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.265036193Z level=info msg="Migration successfully executed" id="Add missing user data" duration=184.356µs 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.265472011Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.266209969Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=737.634µs 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.266611754Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.26711742Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=505.738µs 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.269667907Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.270425249Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=757.057µs 2024-09-21T11:54:43.378 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.27076291Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.276168972Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.404168ms 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.276575968Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.277328529Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=750.975µs 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.277695829Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.27785526Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=159.375µs 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.28059044Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.281144898Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=554.234µs 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.281579926Z level=info msg="Executing migration" id="update login field with orgid to allow for multiple service accounts with same name across orgs" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.281783359Z level=info msg="Migration successfully executed" id="update login field with orgid to allow for multiple service accounts with same name across orgs" duration=203.508µs 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.282195783Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.282708782Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=512.754µs 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.283117635Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.283568341Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=451.311µs 2024-09-21T11:54:43.379 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.283915851Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.284350249Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=434.116µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.284717312Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.285167665Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=450.109µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.285546474Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.285993054Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=446.497µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.286314408Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.286329323Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=15.471µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.286720121Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.287167145Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=446.819µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.287491269Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.287942153Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=450.572µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.288269587Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.28869272Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=422.942µs 2024-09-21T11:54:43.380 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.289035296Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.289456832Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=421.241µs 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.28977706Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.291734891Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=1.957552ms 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.292622585Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.293182007Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=559.111µs 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.293522964Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.293977076Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=453.833µs 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.294293962Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.29474404Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=449.738µs 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.295079692Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.29552412Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=444.051µs 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.29793292Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.298452669Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=519.388µs 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.298806848Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-21T11:54:43.381 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.299058031Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=250.018µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.299371504Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.299715003Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=342.411µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.300051797Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.300292832Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=240.751µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.300662825Z level=info msg="Executing migration" id="create star table" 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.301103131Z level=info msg="Migration successfully executed" id="create star table" duration=424.315µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.301441446Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.301904896Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=463.098µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.302237665Z level=info msg="Executing migration" id="create org table v1" 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.302708023Z level=info msg="Migration successfully executed" id="create org table v1" duration=469.998µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.30305983Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.303525087Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=449.31µs 2024-09-21T11:54:43.382 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.30388073Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.304327533Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=446.588µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.304699789Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.30523483Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=535.268µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.307498058Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.3080189Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=520.432µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.310259849Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.310755783Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=495.878µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.313019777Z level=info msg="Executing migration" id="Update org table charset" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.313040081Z level=info msg="Migration successfully executed" id="Update org table charset" duration=20.441µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.316056544Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.316074825Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=18.261µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.318963491Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.31907969Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=116.108µs 2024-09-21T11:54:43.383 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.321776661Z level=info msg="Executing migration" id="create dashboard table" 2024-09-21T11:54:43.384 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.322307454Z level=info msg="Migration successfully executed" id="create dashboard table" duration=529.655µs 2024-09-21T11:54:43.384 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.324352869Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-21T11:54:43.384 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.324875815Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=522.749µs 2024-09-21T11:54:43.384 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.327048452Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-21T11:54:43.384 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.327612487Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=563.749µs 2024-09-21T11:54:43.385 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.330159905Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-21T11:54:43.385 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.330660194Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=502.164µs 2024-09-21T11:54:43.385 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.332800683Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-21T11:54:43.385 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.333438397Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=654.271µs 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.33552982Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.336098094Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=570.134µs 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.337952838Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.341349399Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.396368ms 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.341714508Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.342232106Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=517.442µs 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.344622187Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.345222926Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=602.39µs 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.347059539Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.34757721Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=517.543µs 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.34973497Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.349977409Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=242.41µs 2024-09-21T11:54:43.386 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.352452273Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.35309967Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=647.369µs 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.354948487Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.354999103Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=49.07µs 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.357335741Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.358476913Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.140786ms 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.360101019Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.361204865Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.103506ms 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.362764647Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.363872826Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.107911ms 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.365323691Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.365842475Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=517.292µs 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.368109798Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.369226877Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.116783ms 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.370971267Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-21T11:54:43.387 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.371510654Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=536.511µs 2024-09-21T11:54:43.388 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.373347392Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-21T11:54:43.388 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.373919197Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=571.331µs 2024-09-21T11:54:43.623 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.3760623Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-21T11:54:43.623 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.376078149Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=17.543µs 2024-09-21T11:54:43.623 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.376520121Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-21T11:54:43.623 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.376535544Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=16.099µs 2024-09-21T11:54:43.623 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.376991033Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-21T11:54:43.623 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.378314124Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.322767ms 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.37868926Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.379870607Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.181013ms 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.380241531Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.381432873Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.191055ms 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.381830272Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.383015377Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.184821ms 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.3833796Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.383542939Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=163.412µs 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.384233207Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-21T11:54:43.624 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.384797995Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=563.567µs 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.385276627Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.385846048Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=569.911µs 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.386214332Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.386270337Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=55.739µs 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.387058586Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.387700571Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=642.121µs 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.38811447Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.388562317Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=448.238µs 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.389001363Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.392675059Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.673827ms 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.393041012Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-21T11:54:43.625 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.393491683Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=450.658µs 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.393908806Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.394383699Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=474.766µs 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.394775676Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.395259006Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=482.912µs 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.395643342Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.395876242Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=233.242µs 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.396452385Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.396823697Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=371.493µs 2024-09-21T11:54:43.626 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.397181684Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.398483662Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.299178ms 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.398844798Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.399325381Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=478.46µs 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.399672857Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.399777901Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=104.948µs 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.400522133Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.400629994Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=108.42µs 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.401505182Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.402059637Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=556.051µs 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.402391277Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.403633755Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.242073ms 2024-09-21T11:54:43.627 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.404013078Z level=info msg="Executing migration" id="create data_source table" 2024-09-21T11:54:43.628 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.40459934Z level=info msg="Migration successfully executed" id="create data_source table" duration=585.961µs 2024-09-21T11:54:43.628 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.404980725Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-21T11:54:43.628 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.405458457Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=477.458µs 2024-09-21T11:54:43.628 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.405823928Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.406305509Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=481.366µs 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.40665726Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.407129277Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=471.901µs 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.40753973Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.408016334Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=476.279µs 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.408393971Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-21T11:54:43.630 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.411743388Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.348988ms 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.412118996Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.412681935Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=562.6µs 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.413032758Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.413528345Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=495.349µs 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.413878289Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.414388543Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=508.299µs 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.414742003Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.415123852Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=381.431µs 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.415583755Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.416975969Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.391797ms 2024-09-21T11:54:43.631 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.418547396Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.419918947Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.371104ms 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.421322913Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.421338433Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=17.168µs 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.423884173Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.424013882Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=129.316µs 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.426679938Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.428110215Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.430012ms 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.429508004Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.429680814Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=174.746µs 2024-09-21T11:54:43.632 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.432000004Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.432116542Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=115.639µs 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.43476434Z level=info msg="Executing migration" id="Add uid column" 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.436215004Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.449358ms 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.437492242Z level=info msg="Executing migration" id="Update uid value" 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.437631135Z level=info msg="Migration successfully executed" id="Update uid value" duration=138.801µs 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.440199608Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.440729457Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=529.362µs 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.442944934Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-21T11:54:43.633 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.443437728Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=492.505µs 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.44581792Z level=info msg="Executing migration" id="create api_key table" 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.446352673Z level=info msg="Migration successfully executed" id="create api_key table" duration=533.884µs 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.448482371Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.448983294Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=500.721µs 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.451302276Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.451784539Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=482.121µs 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.454070922Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.454586177Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=513.184µs 2024-09-21T11:54:43.634 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.457144327Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.45763665Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=492.17µs 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.459917176Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.460498827Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=581.511µs 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.462646095Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.463175696Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=531.376µs 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.465295716Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.469124642Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=3.830602ms 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.469477327Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.469972313Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=494.786µs 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.472231179Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-21T11:54:43.635 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.472724883Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=507.066µs 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.474753839Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.47525751Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=503.473µs 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.477382719Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.477903015Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=519.85µs 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.478261786Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.478516164Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=254.223µs 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.481069069Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.481466734Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=397.475µs 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.48350435Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.483535635Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=31.871µs 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.486110694Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-21T11:54:43.636 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.487599468Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.488289ms 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.487985837Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.489465738Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.479712ms 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.49087337Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.490980246Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=107.186µs 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.49358752Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.49515448Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.566646ms 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.496223199Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.497837576Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.618032ms 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.498193887Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.4986791Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=485.132µs 2024-09-21T11:54:43.637 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.501035205Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.501398396Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=364.11µs 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.504069354Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.504662295Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=594.899µs 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.506704799Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.507291156Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=590.569µs 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.5076444Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.508194384Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=549.664µs 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.508530277Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.509031183Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=500.675µs 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.50936471Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.509401714Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=37.337µs 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.510143254Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-21T11:54:43.638 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.510174822Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=32.017µs 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.510569789Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.512195337Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.625152ms 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.512548368Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.514201385Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.652683ms 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.514561921Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.514598392Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=37.022µs 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.515352129Z level=info msg="Executing migration" id="create quota table v1" 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.515833714Z level=info msg="Migration successfully executed" id="create quota table v1" duration=482.322µs 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.51616823Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.51665038Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=481.761µs 2024-09-21T11:54:43.639 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.516986135Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.517000675Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=15.189µs 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.517832535Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.518325558Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=492.837µs 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.518685061Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.519199665Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=514.805µs 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.519536992Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.521232622Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.695401ms 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.521575293Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.52159056Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=15.809µs 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.52238579Z level=info msg="Executing migration" id="create session table" 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.522949458Z level=info msg="Migration successfully executed" id="create session table" duration=563.339µs 2024-09-21T11:54:43.640 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.523300307Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.52335677Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=56.624µs 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.523728974Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.52378194Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=51.765µs 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.524520074Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.524991652Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=472.471µs 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.525368037Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.525840684Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=472.313µs 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.52618137Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.526195515Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=14.703µs 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.526933284Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-21T11:54:43.641 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.526947013Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=14.278µs 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.527784022Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.529487435Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.702997ms 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.529843072Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.531563767Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.720475ms 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.531906981Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.531969191Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=62.303µs 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.532342513Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.532410788Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=68.737µs 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.533106442Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.533628443Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=521.728µs 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.533982608Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-21T11:54:43.642 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.533996936Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=14.97µs 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.534366285Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.536159617Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.793797ms 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.536505648Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.536598218Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=91.773µs 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.537015077Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.538827825Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.812507ms 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.53916205Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.540994117Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=1.831844ms 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.54131659Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-21T11:54:43.643 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.541353294Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=37.179µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.542204401Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.542829826Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=625.05µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.543175427Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.543691369Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=516.611µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.544045715Z level=info msg="Executing migration" id="create alert table v1" 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.544750136Z level=info msg="Migration successfully executed" id="create alert table v1" duration=704.164µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.545125453Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.545713186Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=586.989µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.547928852Z level=info msg="Executing migration" id="add index alert state" 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.54847116Z level=info msg="Migration successfully executed" id="add index alert state" duration=544.049µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.54881886Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.549344731Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=525.646µs 2024-09-21T11:54:43.644 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.551707347Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.552253313Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=548.126µs 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.554234464Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.554802165Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=567.214µs 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.557056859Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.557601211Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=546.415µs 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.557916871Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.563552107Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.634961ms 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.563901441Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.564357147Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=455.547µs 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.566833067Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.567468829Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=637.375µs 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.569304485Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.569506342Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=201.634µs 2024-09-21T11:54:43.645 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.569815944Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.570171108Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=355.156µs 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.572681135Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.573211838Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=530.316µs 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.575408818Z level=info msg="Executing migration" id="Add column is_default" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.577391218Z level=info msg="Migration successfully executed" id="Add column is_default" duration=1.982959ms 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.578178022Z level=info msg="Executing migration" id="Add column frequency" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.580355738Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.177448ms 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.580698211Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.582903187Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.204876ms 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.583272741Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.585321989Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.04935ms 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.586047782Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-21T11:54:43.646 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.586562415Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=514.412µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.588962281Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.5889995Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=38.926µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.589475202Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.589489681Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=14.253µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.592285476Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.592944238Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=661.791µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.593320607Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.593847879Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=526.991µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.595071027Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.595592457Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=521.228µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.597709103Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.598249712Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=542.403µs 2024-09-21T11:54:43.647 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.598600345Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.599118648Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=518.012µs 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.601534621Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.603701453Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.168753ms 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.604054227Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.606279383Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.225755ms 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.606653322Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.606762471Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=109.149µs 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.607149254Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.607653997Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=504.484µs 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.610001621Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.610527214Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=525.628µs 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.612820003Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.614953706Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.133408ms 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.615398619Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-21T11:54:43.648 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.615434973Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=36.573µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.615828469Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.616341152Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=512.474µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.618840872Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.619400705Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=559.492µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.621258949Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.621331536Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=72.839µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.623946316Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.624506543Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=559.972µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.624868936Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.62539363Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=525.275µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.625732714Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.626261626Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=528.631µs 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.6265971Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-21T11:54:43.649 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.627126455Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=527.929µs 2024-09-21T11:54:43.650 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.627477026Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-21T11:54:43.875 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.628173187Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=695.269µs 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.628576893Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.629277209Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=700.46µs 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.629676851Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.629693982Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=18.268µs 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.63045805Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.633117487Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.66086ms 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.633487628Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.634007543Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=519.809µs 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.634337608Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.636620011Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.281458ms 2024-09-21T11:54:43.876 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.636984366Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.637407558Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=423.024µs 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.637754395Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.638307678Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=552.945µs 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.63864465Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.639161357Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=516.576µs 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.639506999Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.645873731Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.365486ms 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.646218243Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.646653146Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=434.553µs 2024-09-21T11:54:43.877 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.646994309Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.647536945Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=542.355µs 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.647897626Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.648084828Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=187.173µs 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.648460193Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.648887314Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=427.3µs 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.649283699Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.64941082Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=127.259µs 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.650110179Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.653323249Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=3.209581ms 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.653775471Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-21T11:54:43.878 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.656093452Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.317764ms 2024-09-21T11:54:43.879 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.656448726Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.656978524Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=529.499µs 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.657315411Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.657836022Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=522.26µs 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.658180729Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.658320839Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=140.008µs 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.659023656Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.661380952Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.356836ms 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.661738004Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.662279911Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=541.674µs 2024-09-21T11:54:43.882 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.662629791Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.662727824Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=97.231µs 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.663482481Z level=info msg="Executing migration" id="Move region to single row" 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.663703357Z level=info msg="Migration successfully executed" id="Move region to single row" duration=220.882µs 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.66434518Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.664882004Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=536.601µs 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.665212604Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.665726129Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=513.257µs 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.666069903Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.666600695Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=530.291µs 2024-09-21T11:54:43.883 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.666933721Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.66745274Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=518.742µs 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.667779097Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.668304555Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=525.263µs 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.668657659Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.669193139Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=535.2µs 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.669533935Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.669572294Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=38.429µs 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.670342389Z level=info msg="Executing migration" id="create test_data table" 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.670904135Z level=info msg="Migration successfully executed" id="create test_data table" duration=561.405µs 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.671249829Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-21T11:54:43.884 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.671759108Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=509.082µs 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.672124908Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.672650935Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=525.645µs 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.673013885Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.673556795Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=542.602µs 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.673889583Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.673999942Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=109.629µs 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.674722903Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.674974315Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=251.75µs 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.675290031Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-21T11:54:43.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.675326529Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=36.598µs 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.67615242Z level=info msg="Executing migration" id="create team table" 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.67663398Z level=info msg="Migration successfully executed" id="create team table" duration=481.201µs 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.677010977Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.677644907Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=630.047µs 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.678998318Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.67953564Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=536.903µs 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.681878523Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.684385348Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.506554ms 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.684735876Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.68487227Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=136.321µs 2024-09-21T11:54:43.886 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.6874651Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.688026194Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=562.291µs 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.690333669Z level=info msg="Executing migration" id="create team member table" 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.690857609Z level=info msg="Migration successfully executed" id="create team member table" duration=523.615µs 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.692967486Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.693516793Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=549.058µs 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.693873017Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.694420372Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=547.122µs 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.696723891Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.697287262Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=561.795µs 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.699548883Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-21T11:54:43.887 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.702289443Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.740002ms 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.70265014Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.705454235Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.804416ms 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.705819471Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.708405009Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.585266ms 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.708773375Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.709401364Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=627.86µs 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.711594429Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.712192518Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=599.171µs 2024-09-21T11:54:43.888 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.71253815Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.713164802Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=626.304µs 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.713527733Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.714114386Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=586.289µs 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.716249462Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.716798894Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=549.152µs 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.719100359Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.719658727Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=573.552µs 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.721900663Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.722480173Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=579.265µs 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.724469645Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-21T11:54:43.889 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.725078938Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=608.958µs 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.727195144Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.727517855Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=322.555µs 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.729912425Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.730045102Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=132.722µs 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.732593053Z level=info msg="Executing migration" id="create tag table" 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.733107076Z level=info msg="Migration successfully executed" id="create tag table" duration=513.754µs 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.735273597Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.735843481Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=570.399µs 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.738056026Z level=info msg="Executing migration" id="create login attempt table" 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.738644219Z level=info msg="Migration successfully executed" id="create login attempt table" duration=590.057µs 2024-09-21T11:54:43.890 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.740762639Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.741369216Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=606.82µs 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.743256998Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.743843184Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=585.943µs 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.746069829Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.754135466Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=8.065345ms 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.754504225Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.754967871Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=463.483µs 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.75703742Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.757586729Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=548.975µs 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.759837585Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.760031703Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=194.028µs 2024-09-21T11:54:43.891 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.762589403Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.763000308Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=410.641µs 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.765290548Z level=info msg="Executing migration" id="create user auth table" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.765769589Z level=info msg="Migration successfully executed" id="create user auth table" duration=478.588µs 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.768079371Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.768766135Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=688.578µs 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.770686368Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.770725047Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=39.922µs 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.77346159Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.776502885Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=3.04275ms 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.776855189Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.779704938Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.849499ms 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.780055053Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-21T11:54:43.892 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.782923781Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=2.868593ms 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.783249788Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.786165252Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=2.915275ms 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.786508469Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.787074739Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=565.146µs 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.78741636Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.790336674Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.918553ms 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.790718628Z level=info msg="Executing migration" id="create server_lock table" 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.791225123Z level=info msg="Migration successfully executed" id="create server_lock table" duration=506.386µs 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.791575843Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.792138751Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=562.656µs 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.792484186Z level=info msg="Executing migration" id="create user auth token table" 2024-09-21T11:54:43.893 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.793031704Z level=info msg="Migration successfully executed" id="create user auth token table" duration=547.361µs 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.793449201Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.794096117Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=647.081µs 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.794471596Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.795069316Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=597.541µs 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.795426509Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.796131813Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=705.042µs 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.796503015Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.799639067Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.13584ms 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.800036192Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.800628873Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=591.563µs 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.800991925Z level=info msg="Executing migration" id="create cache_data table" 2024-09-21T11:54:43.894 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.801535737Z level=info msg="Migration successfully executed" id="create cache_data table" duration=543.568µs 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.801891823Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.802464295Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=572.248µs 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.802817868Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.803381066Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=562.915µs 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.80376829Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.804406929Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=638.513µs 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.804743535Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.804794771Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=39.772µs 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.805176085Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-21T11:54:43.895 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.805230613Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=53.145µs 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.805568572Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.806124394Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=555.461µs 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.806500816Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.807108178Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=606.933µs 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.807469644Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.808083582Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=613.439µs 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.808417584Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.808455463Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=37.975µs 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.808836522Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-21T11:54:43.896 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.809393085Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=556.262µs 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.809713445Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.810286001Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=572.407µs 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.810626236Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.811234972Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=610.127µs 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.811581174Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.812192046Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=610.622µs 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.812529534Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.815830211Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.300332ms 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.816157844Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.81683692Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=678.698µs 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.817183212Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.817241033Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=58.099µs 2024-09-21T11:54:43.897 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.817947119Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.818529475Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=581.654µs 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.81889869Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.81952955Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=630.622µs 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.81987842Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.820508821Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=628.621µs 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.820852472Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.820893772Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=41.256µs 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.821661987Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.822334437Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=672.068µs 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.822699699Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-21T11:54:43.898 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.823318744Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=618.788µs 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.82367191Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.824282107Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=609.928µs 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.824630111Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.825230693Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=598.623µs 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.825566298Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.828896041Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.329514ms 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.829223797Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.829784498Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=560.67µs 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.830119892Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.830677526Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=557.512µs 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.831015198Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-21T11:54:43.899 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.848457693Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=17.442215ms 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.84886207Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.864915061Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=16.052789ms 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.865287231Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.865909724Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=622.143µs 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.866265007Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.866878157Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=612.927µs 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.867229812Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.870423213Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.193082ms 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.870777485Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.873993716Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.215888ms 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.874370628Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-21T11:54:43.900 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.875006899Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=635.871µs 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.875367672Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.875997308Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=629.128µs 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.876354038Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.876977141Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=624.731µs 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.877342264Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.878037336Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=694.68µs 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.87841012Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.878458022Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=47.73µs 2024-09-21T11:54:43.901 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.879280651Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-21T11:54:44.125 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.882797569Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=3.516476ms 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.883146508Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.886494445Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.34766ms 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.88685636Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.890227671Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=3.37106ms 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.890590026Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.891175584Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=585.626µs 2024-09-21T11:54:44.126 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.891514851Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.892140178Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=624.932µs 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.892477408Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.895812166Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=3.334425ms 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.896166015Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.899517511Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.351243ms 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.899896661Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.900522896Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=625.988µs 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.902710048Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.906111605Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.401058ms 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.906502069Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.909854529Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.352173ms 2024-09-21T11:54:44.127 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.910212224Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.910259936Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=48.115µs 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.911024265Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.911770083Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=745.393µs 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.912141491Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.912836656Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=697.409µs 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.913190703Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.913887025Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=695.993µs 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.914248881Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.914295927Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=47.347µs 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.915007435Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-21T11:54:44.128 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.918609394Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.601723ms 2024-09-21T11:54:44.129 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.918979259Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-21T11:54:44.129 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.922515276Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.535842ms 2024-09-21T11:54:44.130 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.922861966Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-21T11:54:44.130 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.926356392Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.494376ms 2024-09-21T11:54:44.130 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.926714659Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-21T11:54:44.130 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.930241081Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.524871ms 2024-09-21T11:54:44.130 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.930583752Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.934060642Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.476599ms 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.934430705Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.934477856Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=47.258µs 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.93541559Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.936026962Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=613.213µs 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.936409153Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.940019613Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.608405ms 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.940364167Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.940411577Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=47.793µs 2024-09-21T11:54:44.131 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.941306538Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.945089906Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.784854ms 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.94545717Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.946069575Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=612.182µs 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.946454622Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.95009865Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.643694ms 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.950490163Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.95099382Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=503.353µs 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.951399124Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.952007428Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=608.063µs 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.95244446Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.956098987Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.652686ms 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.956499284Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.957001743Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=502.203µs 2024-09-21T11:54:44.132 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.957432353Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.958051269Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=618.616µs 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.958452744Z level=info msg="Executing migration" id="create alert_image table" 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.958970619Z level=info msg="Migration successfully executed" id="create alert_image table" duration=517.538µs 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.959375808Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.959979943Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=603.957µs 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.960329931Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.96037714Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=47.442µs 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.961111509Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.961722308Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=610.584µs 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.962189838Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.962783351Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=593.298µs 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.963139034Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-21T11:54:44.133 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.963371234Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.964113732Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.964552118Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=439.104µs 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.9648983Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.96550399Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=605.222µs 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.965864564Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.969698384Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.833297ms 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.970104456Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.970811254Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=706.434µs 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.971187028Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.971847706Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=660.26µs 2024-09-21T11:54:44.134 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.972195825Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.972730929Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=533.932µs 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.9731185Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.973765731Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=646.819µs 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.974126145Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.974738114Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=611.543µs 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.976959307Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.976978307Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=19.104µs 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.979764519Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.979836052Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=71.519µs 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.982459659Z level=info msg="Executing migration" id="add library_element folder uid" 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.986561566Z level=info msg="Migration successfully executed" id="add library_element folder uid" duration=4.102133ms 2024-09-21T11:54:44.135 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.986925199Z level=info msg="Executing migration" id="populate library_element folder_uid" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.987174746Z level=info msg="Migration successfully executed" id="populate library_element folder_uid" duration=249.334µs 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.989689613Z level=info msg="Executing migration" id="add index library_element org_id-folder_uid-name-kind" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.990388033Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_uid-name-kind" duration=698.022µs 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.992354152Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.992555427Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=201.61µs 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.994946981Z level=info msg="Executing migration" id="create data_keys table" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.995593226Z level=info msg="Migration successfully executed" id="create data_keys table" duration=645.889µs 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.99757482Z level=info msg="Executing migration" id="create secrets table" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:43 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:43.998164689Z level=info msg="Migration successfully executed" id="create secrets table" duration=589.568µs 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.000520889Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.02284544Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=22.326857ms 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.023218628Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-21T11:54:44.136 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.02732875Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.109724ms 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.027692233Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.027800767Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=108.71µs 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.030358068Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.051051093Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=20.690012ms 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.051449518Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.072723528Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=21.273476ms 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.073109262Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.073662008Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=552.416µs 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.075972122Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.076661032Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=688.73µs 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.078861009Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.078995071Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=135.977µs 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.081825897Z level=info msg="Executing migration" id="create permission table" 2024-09-21T11:54:44.137 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.082373407Z level=info msg="Migration successfully executed" id="create permission table" duration=567.724µs 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.084485571Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.085128897Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=643.091µs 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.087077324Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.08771887Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=641.087µs 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.089979785Z level=info msg="Executing migration" id="create role table" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.090537322Z level=info msg="Migration successfully executed" id="create role table" duration=557.286µs 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.092446053Z level=info msg="Executing migration" id="add column display_name" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.096942467Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.495947ms 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.0973499Z level=info msg="Executing migration" id="add column group_name" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.101855526Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.505345ms 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.102228162Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.102865512Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=636.999µs 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.105054151Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.105709283Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=654.859µs 2024-09-21T11:54:44.138 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.107696916Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.108395238Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=698.004µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.110181416Z level=info msg="Executing migration" id="create team role table" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.110859775Z level=info msg="Migration successfully executed" id="create team role table" duration=678.239µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.112880063Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.113540997Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=660.686µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.115873705Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.116565846Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=691.874µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.118537842Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.119204983Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=666.851µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.120995364Z level=info msg="Executing migration" id="create user role table" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.121537993Z level=info msg="Migration successfully executed" id="create user role table" duration=542.515µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.123668484Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.124327656Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=658.898µs 2024-09-21T11:54:44.139 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.126414645Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-21T11:54:44.140 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.127212968Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=798µs 2024-09-21T11:54:44.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-mon[33333]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:44.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.423 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-mon[33333]: Deploying daemon node-exporter.a on smithi107 2024-09-21T11:54:44.423 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.129210993Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.129997839Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=786.054µs 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.131920754Z level=info msg="Executing migration" id="create builtin role table" 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.132607959Z level=info msg="Migration successfully executed" id="create builtin role table" duration=686.942µs 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.134611804Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.135302059Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=688.578µs 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.13571085Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.136360616Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=651.008µs 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.136705898Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.141221133Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.515114ms 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.141563728Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-21T11:54:44.424 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.142199498Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=635.541µs 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.142547535Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.14320036Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=652.516µs 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.143540316Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.144175212Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=634.645µs 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.144495305Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.145128087Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=632.524µs 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.145441592Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.145933804Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=491.863µs 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.146260978Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.146933619Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=672.343µs 2024-09-21T11:54:44.425 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.147261435Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.151901526Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.639768ms 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.152241273Z level=info msg="Executing migration" id="permission kind migration" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.157141417Z level=info msg="Migration successfully executed" id="permission kind migration" duration=4.89984ms 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.157488777Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.162042381Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.553508ms 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.1624203Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.16676465Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.344053ms 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.167124314Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.167781336Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=654.361µs 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.168123407Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.168775184Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=650.043µs 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.169126063Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-21T11:54:44.426 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.169758199Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=631.996µs 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.170109289Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.170666024Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=556.429µs 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.171040489Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.171679228Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=637.295µs 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.172018173Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.172055341Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=37.352µs 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.173009815Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.173027568Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=18.452µs 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.173673596Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-21T11:54:44.427 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.173930538Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=256.975µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.174608966Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.175103445Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=507.446µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.175471673Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.175870814Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=399.083µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.176293539Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.176421452Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=127.842µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.177138352Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.177412385Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=273.868µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.177902129Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.178388309Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=485.837µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.178745838Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.179456107Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=709.934µs 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.179832549Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-21T11:54:44.428 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.184445398Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.612573ms 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.184795604Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.184833539Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=38.292µs 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.185641554Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.186253428Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=611.527µs 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.186594883Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.187277903Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=682.004µs 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.187650662Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.188321541Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=670.762µs 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.18865673Z level=info msg="Executing migration" id="add correlation config column" 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.193334524Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.677689ms 2024-09-21T11:54:44.429 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.193676368Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-21T11:54:44.430 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.194321031Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=644.498µs 2024-09-21T11:54:44.430 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.194660081Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-21T11:54:44.430 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.1953039Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=643.669µs 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.195652282Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.212374075Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=16.721446ms 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.212818127Z level=info msg="Executing migration" id="create correlation v2" 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.213527176Z level=info msg="Migration successfully executed" id="create correlation v2" duration=708.841µs 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.215673287Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.21633299Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=659.423µs 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.218285262Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.21898353Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=697.812µs 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.221050002Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.221704372Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=654.03µs 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.223721104Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.22392514Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=204.017µs 2024-09-21T11:54:44.431 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.226422242Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.226967199Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=544.632µs 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.229027752Z level=info msg="Executing migration" id="add provisioning column" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.233741842Z level=info msg="Migration successfully executed" id="add provisioning column" duration=4.713798ms 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.23409903Z level=info msg="Executing migration" id="create entity_events table" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.234607979Z level=info msg="Migration successfully executed" id="create entity_events table" duration=507.264µs 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.236866338Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.237502272Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=634.051µs 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.23970709Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.23997594Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.242572669Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.24285248Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.24519944Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-21T11:54:44.432 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.245746743Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=547.05µs 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.247912928Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.248539107Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=625.922µs 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.250760134Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.25142596Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=665.603µs 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.253598315Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.254267517Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=668.92µs 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.256403446Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.257073855Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=670.255µs 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.259196392Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.259855754Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=659.109µs 2024-09-21T11:54:44.433 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.260190707Z level=info msg="Executing migration" id="Drop public config table" 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.260727011Z level=info msg="Migration successfully executed" id="Drop public config table" duration=536.027µs 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.263089718Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.263809628Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=719.322µs 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.265864499Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.266545441Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=680.73µs 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.268500366Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.269188718Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=688.147µs 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.271268567Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-21T11:54:44.434 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.271926531Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=657.675µs 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.274020996Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.288626676Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=14.604851ms 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.289036951Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.294252842Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.21568ms 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.294603905Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.299531953Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.927847ms 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.29988792Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.30002909Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=142.691µs 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.302657592Z level=info msg="Executing migration" id="add share column" 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.307762817Z level=info msg="Migration successfully executed" id="add share column" duration=5.104888ms 2024-09-21T11:54:44.435 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.308128627Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.308257646Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=128.958µs 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.31114874Z level=info msg="Executing migration" id="create file table" 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.311811279Z level=info msg="Migration successfully executed" id="create file table" duration=646.135µs 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.312169679Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.312859761Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=689.711µs 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.313861897Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.314596403Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=740.627µs 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.316376228Z level=info msg="Executing migration" id="create file_meta table" 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.316946788Z level=info msg="Migration successfully executed" id="create file_meta table" duration=570.62µs 2024-09-21T11:54:44.436 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.319190198Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.319990938Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=800.287µs 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.321930579Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.322028291Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=98.929µs 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.324589828Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.324684586Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=95.461µs 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.326928496Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.327228295Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=298.75µs 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.329707657Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.329854865Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=146.859µs 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.33224351Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-21T11:54:44.437 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.333051285Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=807.341µs 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.334764616Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.339950585Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=5.185499ms 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.340294852Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.340394987Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=100.288µs 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.342970584Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.343638721Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=668.173µs 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.345816349Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.34604057Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=224.761µs 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.348406347Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-21T11:54:44.438 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.348570081Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=165.64µs 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.351111731Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.351443898Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=333.958µs 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.353691605Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.358593722Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=4.901894ms 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.35894708Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.36439316Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=5.445939ms 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.364760794Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.365419532Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=658.537µs 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.365744593Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.421104096Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=55.359681ms 2024-09-21T11:54:44.439 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.421548321Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-21T11:54:44.440 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.422343179Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=796.896µs 2024-09-21T11:54:44.440 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.422685248Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-21T11:54:44.440 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.42339608Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=710.529µs 2024-09-21T11:54:44.440 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.423809321Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-21T11:54:44.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[41097]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[41097]: Deploying daemon node-exporter.a on smithi107 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[30293]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:44 smithi107 ceph-mon[30293]: Deploying daemon node-exporter.a on smithi107 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.443373899Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=19.564186ms 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.443779124Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.448863962Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.084574ms 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.449241094Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.449434654Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=194.506µs 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.450077081Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.450190101Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=113.92µs 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.450925917Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.451058971Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=132.778µs 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.451778937Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.451917042Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=137.894µs 2024-09-21T11:54:44.681 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.452771713Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-21T11:54:44.682 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.453013206Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=244.88µs 2024-09-21T11:54:44.682 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.453490794Z level=info msg="Executing migration" id="create folder table" 2024-09-21T11:54:44.682 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.454153443Z level=info msg="Migration successfully executed" id="create folder table" duration=663.033µs 2024-09-21T11:54:44.682 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.454557987Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-21T11:54:44.682 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.4552738Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=715.947µs 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.455644659Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.456404622Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=759.51µs 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.456778887Z level=info msg="Executing migration" id="Update folder title length" 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.456807505Z level=info msg="Migration successfully executed" id="Update folder title length" duration=29.055µs 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.457767695Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.458536441Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=768.848µs 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.45892187Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.459623402Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=702.077µs 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.459988954Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-21T11:54:44.683 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.460684439Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=695.636µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.461041153Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.461333497Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=292.349µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.461924016Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.462101686Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=177.769µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.462807459Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.463473631Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=665.95µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.463855978Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.46454197Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=686.143µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.464902038Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.465625345Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=723.737µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.466000643Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.466821827Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=821.234µs 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.46718778Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-21T11:54:44.684 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.46785828Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=670.398µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.468198057Z level=info msg="Executing migration" id="create anon_device table" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.468744619Z level=info msg="Migration successfully executed" id="create anon_device table" duration=546.862µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.469138427Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.46989149Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=752.781µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.470272555Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.470965846Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=693.441µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.47136553Z level=info msg="Executing migration" id="create signing_key table" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.471914956Z level=info msg="Migration successfully executed" id="create signing_key table" duration=547.802µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.472283805Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.472999871Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=715.871µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.473365884Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.474082099Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=716.734µs 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.474440577Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-21T11:54:44.685 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.474618037Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=178.324µs 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.475310584Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.480726482Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.416029ms 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.481126909Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.481594992Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=470.61µs 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.482008027Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.482018824Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=11.197µs 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.482898118Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.483585714Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=687.957µs 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.483948685Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.48396115Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=12.955µs 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.486936432Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.487854464Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=921.106µs 2024-09-21T11:54:44.686 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.488222805Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.488950587Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=727.882µs 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.491161889Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.491853756Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=691.507µs 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.493915163Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.494406519Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=491.357µs 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.497023276Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.497311038Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=291.374µs 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.499569435Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.499607027Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=37.847µs 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.500013237Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.505185992Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.173028ms 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.505546574Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.510656104Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.109291ms 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.511016973Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-21T11:54:44.687 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.511228272Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=211.235µs 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=migrator t=2024-09-21T11:54:44.513875699Z level=info msg="migrations completed" performed=551 skipped=0 duration=1.268375756s 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore t=2024-09-21T11:54:44.514724426Z level=info msg="Created default organization" 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=secrets t=2024-09-21T11:54:44.515268103Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=plugin.store t=2024-09-21T11:54:44.53125568Z level=info msg="Loading plugins..." 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=local.finder t=2024-09-21T11:54:44.572370373Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=plugin.store t=2024-09-21T11:54:44.572388101Z level=info msg="Plugins loaded" count=55 duration=41.135407ms 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=query_data t=2024-09-21T11:54:44.574402109Z level=info msg="Query Service initialization" 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=live.push_http t=2024-09-21T11:54:44.576696339Z level=info msg="Live Push Gateway initialization" 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.migration t=2024-09-21T11:54:44.577770829Z level=info msg=Starting 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.migration t=2024-09-21T11:54:44.578066748Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.migration orgID=1 t=2024-09-21T11:54:44.578378312Z level=info msg="Migrating alerts for organisation" 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.migration orgID=1 t=2024-09-21T11:54:44.578826924Z level=info msg="Alerts found to migrate" alerts=0 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.migration t=2024-09-21T11:54:44.580066799Z level=info msg="Completed alerting migration" 2024-09-21T11:54:44.688 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.state.manager t=2024-09-21T11:54:44.59056338Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=infra.usagestats.collector t=2024-09-21T11:54:44.591863398Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=provisioning.datasources t=2024-09-21T11:54:44.593510024Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=provisioning.datasources t=2024-09-21T11:54:44.605479578Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=provisioning.alerting t=2024-09-21T11:54:44.617920869Z level=info msg="starting to provision alerting" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=provisioning.alerting t=2024-09-21T11:54:44.61793206Z level=info msg="finished to provision alerting" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=grafanaStorageLogger t=2024-09-21T11:54:44.618129511Z level=info msg="Storage starting" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.state.manager t=2024-09-21T11:54:44.618126981Z level=info msg="Warming state cache for startup" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=http.server t=2024-09-21T11:54:44.619808638Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=http.server t=2024-09-21T11:54:44.620058615Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.multiorg.alertmanager t=2024-09-21T11:54:44.618146336Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore.transactions t=2024-09-21T11:54:44.62923783Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=provisioning.dashboard t=2024-09-21T11:54:44.63715078Z level=info msg="starting to provision dashboards" 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.state.manager t=2024-09-21T11:54:44.663092902Z level=info msg="State cache has been initialized" states=0 duration=44.964127ms 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ngalert.scheduler t=2024-09-21T11:54:44.663219726Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-21T11:54:44.689 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ticker t=2024-09-21T11:54:44.663309957Z level=info msg=starting first_tick=2024-09-21T11:54:50Z 2024-09-21T11:54:44.690 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore.transactions t=2024-09-21T11:54:44.66773028Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-21T11:54:44.690 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore.transactions t=2024-09-21T11:54:44.679459254Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-21T11:54:44.934 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore.transactions t=2024-09-21T11:54:44.690231477Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-21T11:54:44.934 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=plugins.update.checker t=2024-09-21T11:54:44.701631581Z level=info msg="Update check succeeded" duration=72.529365ms 2024-09-21T11:54:44.934 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=grafana.update.checker t=2024-09-21T11:54:44.712992041Z level=info msg="Update check succeeded" duration=83.954767ms 2024-09-21T11:54:44.934 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=provisioning.dashboard t=2024-09-21T11:54:44.813675426Z level=info msg="finished to provision dashboards" 2024-09-21T11:54:44.935 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=grafana-apiserver t=2024-09-21T11:54:44.933002615Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-21T11:54:45.004 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:44 smithi107 bash[75848]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-21T11:54:45.300 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:54:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=grafana-apiserver t=2024-09-21T11:54:44.933503833Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-21T11:54:46.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:46 smithi107 ceph-mon[41097]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:46 smithi107 ceph-mon[30293]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:46.754 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:46 smithi107 bash[75848]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-21T11:54:46.754 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:46 smithi107 bash[75848]: Writing manifest to image destination 2024-09-21T11:54:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:46 smithi193 ceph-mon[33333]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:47.170 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:46 smithi107 podman[75848]: 2024-09-21 11:54:46.817273652 +0000 UTC m=+2.684927932 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-21T11:54:47.170 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:46 smithi107 podman[75848]: 2024-09-21 11:54:46.924473506 +0000 UTC m=+2.792127776 container create 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:47.170 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 podman[75848]: 2024-09-21 11:54:47.169368733 +0000 UTC m=+3.037023004 container init 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 podman[75848]: 2024-09-21 11:54:47.175373573 +0000 UTC m=+3.043027843 container start 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.178Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.178Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.179Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.179Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-21T11:54:47.504 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=arp 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-21T11:54:47.505 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=edac 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-21T11:54:47.506 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=os 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-21T11:54:47.507 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=stat 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=time 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=uname 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.180Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.181Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a[75923]: ts=2024-09-21T11:54:47.181Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-21T11:54:47.508 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 bash[75848]: 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d 2024-09-21T11:54:47.509 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 11:54:47 smithi107 systemd[1]: Started Ceph node-exporter.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:54:48.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-mon[33333]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:48.475 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.476 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.476 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.476 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-mon[33333]: Deploying daemon node-exporter.b on smithi193 2024-09-21T11:54:48.476 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[30293]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[30293]: Deploying daemon node-exporter.b on smithi193 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[41097]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[41097]: Deploying daemon node-exporter.b on smithi193 2024-09-21T11:54:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:48 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:48.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:54:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:54:48.800 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:48 smithi193 bash[63489]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-21T11:54:49.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:49.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:50.300 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:49 smithi193 bash[63489]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-21T11:54:50.300 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 bash[63489]: Writing manifest to image destination 2024-09-21T11:54:50.626 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-mon[33333]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:50.627 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 podman[63489]: 2024-09-21 11:54:50.299729636 +0000 UTC m=+2.104906815 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-21T11:54:50.627 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 podman[63489]: 2024-09-21 11:54:50.398037834 +0000 UTC m=+2.203215010 container create 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T11:54:50.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:50 smithi107 ceph-mon[41097]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:50 smithi107 ceph-mon[30293]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:51.050 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 podman[63489]: 2024-09-21 11:54:50.625359847 +0000 UTC m=+2.430537033 container init 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 podman[63489]: 2024-09-21 11:54:50.63086963 +0000 UTC m=+2.436046811 container start 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.633Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.633Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.634Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.634Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.634Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.634Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=arp 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-21T11:54:51.051 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=edac 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-21T11:54:51.052 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-21T11:54:51.053 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=os 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=stat 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=time 2024-09-21T11:54:51.054 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=uname 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b[63566]: ts=2024-09-21T11:54:50.635Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 bash[63489]: 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 2024-09-21T11:54:51.055 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 11:54:50 smithi193 systemd[1]: Started Ceph node-exporter.b for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:54:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:51 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:51 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:51 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:51 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:51 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:51 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:52.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:51 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:54:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:53.060 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:52 smithi193 ceph-mon[33333]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:53.061 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:52 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:53.061 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:52 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:53.071 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-mon[41097]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:53.071 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:53.071 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:53.072 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-mon[30293]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:53.072 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:53.072 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:52 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:55.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:54 smithi107 ceph-mon[30293]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:55.040 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:54 smithi107 ceph-mon[41097]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:54 smithi193 ceph-mon[33333]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:56.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:56.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:55 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:56.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:56.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:55 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:54:57.209 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:56 smithi107 ceph-mon[41097]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:57.209 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:56 smithi107 ceph-mon[41097]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-21T11:54:57.210 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:56 smithi107 ceph-mon[41097]: Reconfiguring daemon alertmanager.a on smithi107 2024-09-21T11:54:57.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:56 smithi107 ceph-mon[30293]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:57.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:56 smithi107 ceph-mon[30293]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-21T11:54:57.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:56 smithi107 ceph-mon[30293]: Reconfiguring daemon alertmanager.a on smithi107 2024-09-21T11:54:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:56 smithi193 ceph-mon[33333]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:54:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:56 smithi193 ceph-mon[33333]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-21T11:54:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:56 smithi193 ceph-mon[33333]: Reconfiguring daemon alertmanager.a on smithi107 2024-09-21T11:54:58.753 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:58 smithi107 systemd[1]: Stopping Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:54:58.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:54:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:54:59.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:58 smithi107 ceph-mon[30293]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:59.102 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:58 smithi107 ceph-mon[41097]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:59.102 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:58 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[75594]: ts=2024-09-21T11:54:58.775Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-21T11:54:59.102 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:58 smithi107 podman[76629]: 2024-09-21 11:54:58.796688895 +0000 UTC m=+0.138440102 container died 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:59.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:58 smithi193 ceph-mon[33333]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:54:59.459 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 podman[76629]: 2024-09-21 11:54:59.222713142 +0000 UTC m=+0.564464341 container cleanup 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:59.460 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 bash[76629]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a 2024-09-21T11:54:59.460 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 podman[76644]: 2024-09-21 11:54:59.458036563 +0000 UTC m=+0.656565896 container remove 5211524a14bdd94901587e9588a07de6597b90b090f193d4c85d1960c9012564 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:54:59.753 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 podman[76644]: 2024-09-21 11:54:59.54215089 +0000 UTC m=+0.740680224 volume remove ba4195b000b9130d9db765dccbaf8e4f3edf322e8d18ea2fe41c5323b87423aa 2024-09-21T11:55:00.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:54:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:00.162 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:54:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:00.162 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-21T11:55:00.162 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 systemd[1]: Stopped Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:55:00.162 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:54:59 smithi107 systemd[1]: Starting Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:55:00.162 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 podman[76736]: 2024-09-21 11:54:59.954992451 +0000 UTC m=+0.019170837 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-21T11:55:00.162 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 podman[76736]: 2024-09-21 11:55:00.067980539 +0000 UTC m=+0.132158916 volume create c50b50c27cd2424f744f702ca4c324d099e6326394c6156208b6925367f0e47e 2024-09-21T11:55:00.162 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 podman[76736]: 2024-09-21 11:55:00.160460376 +0000 UTC m=+0.224638753 container create cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:54:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 podman[76736]: 2024-09-21 11:55:00.497629863 +0000 UTC m=+0.561808259 container init cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 podman[76736]: 2024-09-21 11:55:00.504203421 +0000 UTC m=+0.568381798 container start cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.518Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.518Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.520Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.107 port=9094 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.521Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.550Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.551Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:55:00.754 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.553Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-21T11:55:00.755 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:00.553Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-21T11:55:00.755 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 bash[76736]: cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 2024-09-21T11:55:00.755 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:00 smithi107 systemd[1]: Started Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:55:01.237 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:00 smithi193 ceph-mon[33333]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:01.237 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:00 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:01.237 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:00 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:01.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-mon[41097]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:01.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:01.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-mon[30293]: pgmap v29: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:00 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:01 smithi193 ceph-mon[33333]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-21T11:55:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:01 smithi193 ceph-mon[33333]: Reconfiguring daemon prometheus.a on smithi193 2024-09-21T11:55:02.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:01 smithi107 ceph-mon[41097]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-21T11:55:02.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:01 smithi107 ceph-mon[41097]: Reconfiguring daemon prometheus.a on smithi193 2024-09-21T11:55:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:01 smithi107 ceph-mon[30293]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-21T11:55:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:01 smithi107 ceph-mon[30293]: Reconfiguring daemon prometheus.a on smithi193 2024-09-21T11:55:02.523 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:55:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:02.983 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:02.522Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000821025s 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:02 smithi107 ceph-mon[30293]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:03.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:02 smithi107 ceph-mon[41097]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:02 smithi193 ceph-mon[33333]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:04.183 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:03 smithi193 systemd[1]: Stopping Ceph prometheus.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:55:04.550 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.182Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.183Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-21T11:55:04.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.183Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-21T11:55:04.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[61653]: ts=2024-09-21T11:55:04.183Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-21T11:55:04.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:04 smithi193 podman[64399]: 2024-09-21 11:55:04.1925392 +0000 UTC m=+0.136520000 container died 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:05.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:04 smithi107 ceph-mon[30293]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:05.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:04 smithi107 ceph-mon[41097]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:04 smithi193 ceph-mon[33333]: pgmap v31: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:06 smithi193 ceph-mon[33333]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:07.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:06 smithi107 ceph-mon[30293]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:07.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:06 smithi107 ceph-mon[41097]: pgmap v32: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:08.050 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:07 smithi193 podman[64399]: 2024-09-21 11:55:07.77789394 +0000 UTC m=+3.721874731 container cleanup 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:08.050 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:07 smithi193 bash[64399]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a 2024-09-21T11:55:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:07 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:08.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:08.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]} : dispatch 2024-09-21T11:55:08.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:07 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:08.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:55:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:55:08.800 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 podman[64417]: 2024-09-21 11:55:08.513304147 +0000 UTC m=+4.321882060 container remove 110793259c6c0f7c687ec3b40a7dbcf0da8badf22b27aa4d55e3944a70690229 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:08.800 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@prometheus.a.service: Deactivated successfully. 2024-09-21T11:55:08.800 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 systemd[1]: Stopped Ceph prometheus.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:55:08.801 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@prometheus.a.service: Consumed 1.087s CPU time. 2024-09-21T11:55:08.801 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 systemd[1]: Starting Ceph prometheus.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T11:55:09.125 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 podman[64516]: 2024-09-21 11:55:08.767915082 +0000 UTC m=+0.017031046 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-21T11:55:09.125 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:08 smithi193 podman[64516]: 2024-09-21 11:55:08.864045512 +0000 UTC m=+0.113161528 container create 7924e77b724b4fcad71cce2a1ca2870402622af56d6103c9dabf24eecd05cfd5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:09.126 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 podman[64516]: 2024-09-21 11:55:09.100160637 +0000 UTC m=+0.349276588 container init 7924e77b724b4fcad71cce2a1ca2870402622af56d6103c9dabf24eecd05cfd5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:09.126 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 podman[64516]: 2024-09-21 11:55:09.105251789 +0000 UTC m=+0.354367741 container start 7924e77b724b4fcad71cce2a1ca2870402622af56d6103c9dabf24eecd05cfd5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:55:09.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-mon[33333]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:09.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-21T11:55:09.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:55:09.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-21T11:55:09.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-mon[33333]: osdmap e65: 8 total, 8 up, 8 in 2024-09-21T11:55:09.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE Bus STOPPING 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[41097]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[41097]: osdmap e65: 8 total, 8 up, 8 in 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[30293]: pgmap v33: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-09-21T11:55:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-mon[30293]: osdmap e65: 8 total, 8 up, 8 in 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.124Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.124Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.124Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi193 (none))" 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.124Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.124Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.126Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.127Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.129Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-21T11:55:09.551 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.129Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.130Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.130Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.735µs 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.130Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.131Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.131Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.131Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=22.15µs wal_replay_duration=522.383µs wbl_replay_duration=110ns total_replay_duration=577.978µs 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.132Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.132Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.132Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.148Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.474041ms db_storage=734ns remote_storage=1.091µs web_handler=270ns query_engine=696ns scrape=390.366µs scrape_sd=96.497µs notify=9.255µs notify_sd=8.236µs rules=14.553763ms tracing=5.761µs 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.148Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T11:55:09.148Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 bash[64516]: 7924e77b724b4fcad71cce2a1ca2870402622af56d6103c9dabf24eecd05cfd5 2024-09-21T11:55:09.552 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 11:55:09 smithi193 systemd[1]: Started Ceph prometheus.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T11:55:09.799 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:55:09.799 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE Bus STOPPED 2024-09-21T11:55:09.799 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE Bus STARTING 2024-09-21T11:55:09.799 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE Serving on http://:::9283 2024-09-21T11:55:10.254 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE Bus STARTED 2024-09-21T11:55:10.254 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:09 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:09] ENGINE Bus STOPPING 2024-09-21T11:55:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: osdmap e66: 8 total, 8 up, 8 in 2024-09-21T11:55:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T11:55:10.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"} : dispatch 2024-09-21T11:55:10.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T11:55:10.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"} : dispatch 2024-09-21T11:55:10.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T11:55:10.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"} : dispatch 2024-09-21T11:55:10.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:10.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:55:10.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:10.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: osdmap e66: 8 total, 8 up, 8 in 2024-09-21T11:55:10.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T11:55:10.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"} : dispatch 2024-09-21T11:55:10.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T11:55:10.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"} : dispatch 2024-09-21T11:55:10.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T11:55:10.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"} : dispatch 2024-09-21T11:55:10.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:10.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:55:10.296 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:10.296 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: osdmap e66: 8 total, 8 up, 8 in 2024-09-21T11:55:10.296 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.296 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.297 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T11:55:10.297 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"} : dispatch 2024-09-21T11:55:10.297 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.297 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T11:55:10.297 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"} : dispatch 2024-09-21T11:55:10.297 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.298 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T11:55:10.298 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"} : dispatch 2024-09-21T11:55:10.298 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:10.298 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:10.298 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:10 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:55:10.554 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:55:10.555 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Bus STOPPED 2024-09-21T11:55:10.555 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Bus STARTING 2024-09-21T11:55:10.555 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Serving on http://:::9283 2024-09-21T11:55:10.555 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Bus STARTED 2024-09-21T11:55:10.555 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Bus STOPPING 2024-09-21T11:55:10.556 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T11:55:10.525Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003893978s 2024-09-21T11:55:10.911 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:55:11.254 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Bus STOPPED 2024-09-21T11:55:11.254 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:10 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:10] ENGINE Bus STARTING 2024-09-21T11:55:11.254 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:11] ENGINE Serving on http://:::9283 2024-09-21T11:55:11.254 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:11:55:11] ENGINE Bus STARTED 2024-09-21T11:55:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:11.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:11 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T11:55:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T11:55:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T11:55:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:11.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:11 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:55:12.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:12 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:12.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:12 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:12.329 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:12 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:12.329 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:12 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:12.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:12 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:12.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:12 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:55:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:13 smithi193 ceph-mon[33333]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:55:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:13 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:13 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:13.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:13 smithi107 ceph-mon[41097]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:55:13.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:13 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:13.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:13 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:13.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:13 smithi107 ceph-mon[30293]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:55:13.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:13 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:13.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:13 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: Cluster is now healthy 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-21T11:55:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: Cluster is now healthy 2024-09-21T11:55:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:55:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:55:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:14 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: pgmap v38: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:55:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-21T11:55:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: Cluster is now healthy 2024-09-21T11:55:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:55:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:55:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:14 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:17.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:16 smithi107 ceph-mon[41097]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:55:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:16 smithi107 ceph-mon[30293]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:55:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:16 smithi193 ceph-mon[33333]: pgmap v39: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:55:18.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:55:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:55:18.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:18 smithi193 ceph-mon[33333]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 1 objects/s recovering 2024-09-21T11:55:19.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:18 smithi107 ceph-mon[41097]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 1 objects/s recovering 2024-09-21T11:55:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:18 smithi107 ceph-mon[30293]: pgmap v40: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 1 objects/s recovering 2024-09-21T11:55:20.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:21.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:20 smithi107 ceph-mon[41097]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 995 B/s rd, 0 op/s; 21 B/s, 1 objects/s recovering 2024-09-21T11:55:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:20 smithi107 ceph-mon[30293]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 995 B/s rd, 0 op/s; 21 B/s, 1 objects/s recovering 2024-09-21T11:55:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:20 smithi193 ceph-mon[33333]: pgmap v41: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 995 B/s rd, 0 op/s; 21 B/s, 1 objects/s recovering 2024-09-21T11:55:22.737 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:55:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:22.738 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:22 smithi107 ceph-mon[30293]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-09-21T11:55:22.738 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:22 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:23.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:22 smithi107 ceph-mon[41097]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-09-21T11:55:23.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:22 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:22 smithi193 ceph-mon[33333]: pgmap v42: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 1 objects/s recovering 2024-09-21T11:55:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:22 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:25.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:24 smithi107 ceph-mon[41097]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-09-21T11:55:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:24 smithi107 ceph-mon[30293]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-09-21T11:55:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:24 smithi193 ceph-mon[33333]: pgmap v43: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 18 B/s, 1 objects/s recovering 2024-09-21T11:55:27.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:26 smithi107 ceph-mon[41097]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:26 smithi107 ceph-mon[30293]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:26 smithi193 ceph-mon[33333]: pgmap v44: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:28.577 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'. 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-09-21T11:55:28.578 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.579 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T11:55:28.579 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-21T11:55:28.579 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at 5c1cdc3c470 ceph-volume: fix OSD lvm/tpm2 activation 2024-09-21T11:55:28.583 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T11:55:28.606 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T11:55:28.608 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T11:55:28.608 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T11:55:28.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:55:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:55:28.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:28 smithi193 ceph-mon[33333]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:28 smithi107 ceph-mon[41097]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:29.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:28 smithi107 ceph-mon[30293]: pgmap v45: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:55:29.322 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T11:55:29.392 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T11:55:29.431 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T11:55:29.432 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T11:55:29.433 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T11:55:29.470 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T11:55:29.474 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T11:55:29.474 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T11:55:29.529 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-21T11:55:29.531 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-21T11:55:29.531 DEBUG:teuthology.orchestra.run.smithi107:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ parallel=1 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' '' = --serial ']' 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ crimson=0 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' '' = --crimson ']' 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ color= 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -t 1 ']' 2024-09-21T11:55:29.599 INFO:tasks.workunit.client.0.smithi107.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-21T11:55:29.600 INFO:tasks.workunit.client.0.smithi107.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-21T11:55:29.600 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-21T11:55:29.600 INFO:tasks.workunit.client.0.smithi107.stderr:+ declare -A pids 2024-09-21T11:55:29.601 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.601 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.601 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_aio 2024-09-21T11:55:29.601 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_aio' 2024-09-21T11:55:29.601 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_aio 2024-09-21T11:55:29.601 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.602 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_aio 2024-09-21T11:55:29.602 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77370 2024-09-21T11:55:29.602 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_aio on pid 77370' 2024-09-21T11:55:29.603 INFO:tasks.workunit.client.0.smithi107.stdout:test api_aio on pid 77370 2024-09-21T11:55:29.605 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77370 2024-09-21T11:55:29.605 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.605 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-21T11:55:29.605 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.605 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_aio_pp 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_aio_pp' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_aio_pp 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.606 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_aio_pp 2024-09-21T11:55:29.607 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77377 2024-09-21T11:55:29.608 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_aio_pp on pid 77377' 2024-09-21T11:55:29.608 INFO:tasks.workunit.client.0.smithi107.stdout:test api_aio_pp on pid 77377 2024-09-21T11:55:29.610 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77377 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_io 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_io' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_io 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.611 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.612 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.613 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_io 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77386 2024-09-21T11:55:29.614 INFO:tasks.workunit.client.0.smithi107.stdout:test api_io on pid 77386 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_io on pid 77386' 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77386 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_io_pp 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_io_pp' 2024-09-21T11:55:29.619 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_io_pp 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.620 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.621 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.622 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_io_pp 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77399 2024-09-21T11:55:29.623 INFO:tasks.workunit.client.0.smithi107.stdout:test api_io_pp on pid 77399 2024-09-21T11:55:29.628 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_io_pp on pid 77399' 2024-09-21T11:55:29.628 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77399 2024-09-21T11:55:29.628 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.628 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-21T11:55:29.628 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.628 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_asio 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_asio' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_asio 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.629 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.630 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.631 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.632 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.633 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_asio 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77414 2024-09-21T11:55:29.634 INFO:tasks.workunit.client.0.smithi107.stdout:test api_asio on pid 77414 2024-09-21T11:55:29.640 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_asio on pid 77414' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77414 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_list 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_list' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.641 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.642 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.643 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_list 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.644 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77370/exe 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.645 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.646 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.647 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.648 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.649 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_list 2024-09-21T11:55:29.649 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.649 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.649 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77430 2024-09-21T11:55:29.649 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_list on pid 77430' 2024-09-21T11:55:29.649 INFO:tasks.workunit.client.0.smithi107.stdout:test api_list on pid 77430 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77430 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_lock 2024-09-21T11:55:29.654 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_aio: /' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_lock' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.655 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.656 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_lock 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.657 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.658 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77377/exe 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.659 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.660 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.661 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_lock 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.662 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.663 INFO:tasks.workunit.client.0.smithi107.stdout:test api_lock on pid 77451 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77451 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_lock on pid 77451' 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77451 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-21T11:55:29.665 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_lock_pp 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_lock_pp' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.666 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.667 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.668 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_lock_pp 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.669 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77386/exe 2024-09-21T11:55:29.670 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.671 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.672 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.673 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_lock_pp 2024-09-21T11:55:29.674 INFO:tasks.workunit.client.0.smithi107.stdout:test api_lock_pp on pid 77470 2024-09-21T11:55:29.677 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77470 2024-09-21T11:55:29.677 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_lock_pp on pid 77470' 2024-09-21T11:55:29.677 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77470 2024-09-21T11:55:29.677 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.677 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_misc 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_misc' 2024-09-21T11:55:29.678 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_io.log 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_io: /' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.679 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.680 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_misc 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.681 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.682 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.684 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77399/exe 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.685 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_misc 2024-09-21T11:55:29.686 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77490 2024-09-21T11:55:29.687 INFO:tasks.workunit.client.0.smithi107.stdout:test api_misc on pid 77490 2024-09-21T11:55:29.689 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_misc on pid 77490' 2024-09-21T11:55:29.689 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77490 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_misc_pp 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_misc_pp' 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.690 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.691 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.692 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.693 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.694 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_misc_pp 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.695 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77414/exe 2024-09-21T11:55:29.696 INFO:tasks.workunit.client.0.smithi107.stdout:test api_misc_pp on pid 77510 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_misc_pp 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.700 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77510 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_misc_pp on pid 77510' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77510 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.701 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.702 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_tier_pp 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.703 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_tier_pp' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_asio: /' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.704 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.705 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.706 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.707 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_tier_pp 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.708 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77430/exe 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.709 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.710 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.711 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.711 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.711 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.711 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.711 INFO:tasks.workunit.client.0.smithi107.stdout:test api_tier_pp on pid 77533 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_tier_pp 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77533 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_tier_pp on pid 77533' 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77533 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_list: /' 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.715 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_pool 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_pool' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_list.log 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.716 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.717 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.718 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77451/exe 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.719 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.720 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.721 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.722 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_pool 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_lock: /' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.723 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.724 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_pool 2024-09-21T11:55:29.725 INFO:tasks.workunit.client.0.smithi107.stdout:test api_pool on pid 77567 2024-09-21T11:55:29.729 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77567 2024-09-21T11:55:29.729 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_pool on pid 77567' 2024-09-21T11:55:29.729 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77567 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.730 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.731 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_snapshots 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_snapshots' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.732 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.734 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.735 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.736 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77490/exe 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.737 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.738 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.739 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_snapshots 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.740 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77470/exe 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_misc: /' 2024-09-21T11:55:29.741 INFO:tasks.workunit.client.0.smithi107.stdout:test api_snapshots on pid 77635 2024-09-21T11:55:29.745 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_snapshots 2024-09-21T11:55:29.745 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.745 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77635 2024-09-21T11:55:29.745 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_snapshots on pid 77635' 2024-09-21T11:55:29.745 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77635 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_snapshots_pp 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_snapshots_pp' 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.746 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.747 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.748 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.749 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.750 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.751 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.752 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_snapshots_pp 2024-09-21T11:55:29.753 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-21T11:55:29.754 INFO:tasks.workunit.client.0.smithi107.stdout:test api_snapshots_pp on pid 77674 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_snapshots_pp 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77674 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_snapshots_pp on pid 77674' 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77674 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_stat 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_stat' 2024-09-21T11:55:29.757 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_stat 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77510/exe 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.758 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.759 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.760 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.761 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.762 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-21T11:55:29.763 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-09-21T11:55:29.764 INFO:tasks.workunit.client.0.smithi107.stdout:test api_stat on pid 77707 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_stat 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77707 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_stat on pid 77707' 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77707 2024-09-21T11:55:29.766 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.767 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.768 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_stat_pp 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.769 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_stat_pp' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.770 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_stat_pp 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.771 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.772 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77533/exe 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.773 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.774 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_stat_pp 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77752 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_stat_pp on pid 77752' 2024-09-21T11:55:29.775 INFO:tasks.workunit.client.0.smithi107.stdout:test api_stat_pp on pid 77752 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77752 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.779 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.780 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-21T11:55:29.781 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77567/exe 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.782 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.783 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_pool: /' 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-09-21T11:55:29.784 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77635/exe 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.785 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.786 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.787 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_watch_notify 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_watch_notify' 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_watch_notify 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.788 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.789 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.790 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.791 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.791 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.791 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.791 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.791 INFO:tasks.workunit.client.0.smithi107.stdout:test api_watch_notify on pid 77799 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_watch_notify 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77799 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_watch_notify on pid 77799' 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77799 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77674/exe 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.794 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_watch_notify_pp 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_watch_notify_pp' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.795 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.796 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_watch_notify_pp 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.797 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.798 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.799 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.800 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.801 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.802 INFO:tasks.workunit.client.0.smithi107.stdout:test api_watch_notify_pp on pid 77830 2024-09-21T11:55:29.804 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_watch_notify_pp 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77830 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_watch_notify_pp on pid 77830' 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77830 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_cmd 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_cmd' 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_cmd 2024-09-21T11:55:29.805 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.806 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.807 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.808 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.809 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.810 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.811 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.812 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.812 INFO:tasks.workunit.client.0.smithi107.stdout:test api_cmd on pid 77851 2024-09-21T11:55:29.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_cmd 2024-09-21T11:55:29.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77851 2024-09-21T11:55:29.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_cmd on pid 77851' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77851 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.817 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.818 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.819 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.820 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_cmd_pp 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.821 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.822 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_cmd_pp' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.823 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.824 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.824 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.824 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.824 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.825 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.825 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.825 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.825 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.825 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.825 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_cmd_pp 2024-09-21T11:55:29.826 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77707/exe 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.827 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.828 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.829 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.829 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-09-21T11:55:29.829 INFO:tasks.workunit.client.0.smithi107.stdout:test api_cmd_pp on pid 77888 2024-09-21T11:55:29.831 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_cmd_pp 2024-09-21T11:55:29.831 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77888 2024-09-21T11:55:29.831 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_cmd_pp on pid 77888' 2024-09-21T11:55:29.831 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77888 2024-09-21T11:55:29.831 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.831 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_service 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_service' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.832 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.833 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_service 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.834 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.835 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.836 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77799/exe 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.837 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.838 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.839 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.840 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.840 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.840 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.840 INFO:tasks.workunit.client.0.smithi107.stdout:test api_service on pid 77915 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_service 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77915 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_service on pid 77915' 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77915 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.844 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.845 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.846 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_service_pp 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_service_pp' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.847 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_stat: /' 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77752/exe 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.848 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.849 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77830/exe 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.850 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.851 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_service_pp 2024-09-21T11:55:29.852 INFO:tasks.workunit.client.0.smithi107.stdout:test api_service_pp on pid 77956 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_service_pp 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77956 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_service_pp on pid 77956' 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=77956 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.856 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.857 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.858 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_c_write_operations 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_c_write_operations' 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-21T11:55:29.859 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.860 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.861 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.862 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.863 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.864 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.865 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.866 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.867 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.868 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_c_write_operations 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.869 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.870 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.870 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.870 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.870 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77888/exe 2024-09-21T11:55:29.870 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_c_write_operations 2024-09-21T11:55:29.870 INFO:tasks.workunit.client.0.smithi107.stdout:test api_c_write_operations on pid 78025 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78025 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_c_write_operations on pid 78025' 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78025 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.874 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.875 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77851/exe 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_c_read_operations 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_c_read_operations' 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.876 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-21T11:55:29.877 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.878 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_cmd: /' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.879 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_c_read_operations 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.880 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.881 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.882 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.883 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.883 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.883 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.883 INFO:tasks.workunit.client.0.smithi107.stdout:test api_c_read_operations on pid 78078 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_c_read_operations 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78078 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_c_read_operations on pid 78078' 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78078 2024-09-21T11:55:29.888 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s list_parallel 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' list_parallel' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-21T11:55:29.889 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.890 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.891 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77915/exe 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.892 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.893 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.894 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.895 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_service.log 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_service: /' 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/77956/exe 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-09-21T11:55:29.896 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.897 INFO:tasks.workunit.client.0.smithi107.stdout:test list_parallel on pid 78119 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo list_parallel 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=list_parallel 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78119 2024-09-21T11:55:29.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test list_parallel on pid 78119' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78119 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s open_pools_parallel 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.899 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' open_pools_parallel' 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.900 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo open_pools_parallel 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.901 INFO:tasks.workunit.client.0.smithi107.stdout:test open_pools_parallel on pid 78127 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=open_pools_parallel 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78127 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test open_pools_parallel on pid 78127' 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78127 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.904 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.905 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s delete_pools_parallel 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' delete_pools_parallel' 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78025/exe 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.906 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.908 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo delete_pools_parallel 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.909 INFO:tasks.workunit.client.0.smithi107.stdout:test delete_pools_parallel on pid 78162 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=delete_pools_parallel 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.912 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78162 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test delete_pools_parallel on pid 78162' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78162 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.913 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.914 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s cls 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.915 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.916 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' cls' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.917 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo cls 2024-09-21T11:55:29.918 INFO:tasks.workunit.client.0.smithi107.stdout:test cls on pid 78192 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=cls 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78192 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test cls on pid 78192' 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78192 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.922 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.923 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.924 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.925 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s cmd 2024-09-21T11:55:29.926 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' cmd' 2024-09-21T11:55:29.927 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.928 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo cmd 2024-09-21T11:55:29.929 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.930 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.931 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.931 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.931 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.931 INFO:tasks.workunit.client.0.smithi107.stdout:test cmd on pid 78232 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=cmd 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78078/exe 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78232 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test cmd on pid 78232' 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78232 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s handler_error 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' handler_error' 2024-09-21T11:55:29.935 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.936 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.937 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.938 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo handler_error 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.939 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.940 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.941 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.942 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=handler_error 2024-09-21T11:55:29.943 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.944 INFO:tasks.workunit.client.0.smithi107.stdout:test handler_error on pid 78250 2024-09-21T11:55:29.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78250 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test handler_error on pid 78250' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78250 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.950 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.951 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.952 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.953 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.956 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78119/exe 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.957 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.958 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.959 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78127/exe 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ list_parallel: /' 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s io 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' io' 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo io 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78162/exe 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.964 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.965 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.966 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.967 INFO:tasks.workunit.client.0.smithi107.stdout:test io on pid 78316 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=io 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78316 2024-09-21T11:55:29.970 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test io on pid 78316' 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78316 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s ec_io 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' ec_io' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.972 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.973 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78192/exe 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.974 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.975 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.976 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.977 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_cls 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_cls.log 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ cls: /' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo ec_io 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:29.980 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:29.980 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.980 INFO:tasks.workunit.client.0.smithi107.stdout:test ec_io on pid 78389 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=ec_io 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78389 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test ec_io on pid 78389' 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78389 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s list 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' list' 2024-09-21T11:55:29.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:29.985 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.986 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:29.987 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.988 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78316/exe 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:29.989 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_io.log 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_io 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo list 2024-09-21T11:55:29.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ io: /' 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.992 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.993 INFO:tasks.workunit.client.0.smithi107.stdout:test list on pid 78426 2024-09-21T11:55:29.993 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=list 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78426 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test list on pid 78426' 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78426 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:29.994 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s ec_list 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' ec_list' 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:29.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:29.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:29.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:29.999 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:29.999 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.006 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78250/exe 2024-09-21T11:55:30.007 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.007 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.008 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.008 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo ec_list 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.010 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.011 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.011 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.011 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.011 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.011 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.012 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.013 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_handler_error 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ handler_error: /' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.014 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78232/exe 2024-09-21T11:55:30.016 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=ec_list 2024-09-21T11:55:30.016 INFO:tasks.workunit.client.0.smithi107.stdout:test ec_list on pid 78535 2024-09-21T11:55:30.016 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78535 2024-09-21T11:55:30.016 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test ec_list on pid 78535' 2024-09-21T11:55:30.016 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78535 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_handler_error.log 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s misc 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' misc' 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_cmd 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ cmd: /' 2024-09-21T11:55:30.020 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_cmd.log 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo misc 2024-09-21T11:55:30.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.024 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.025 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.025 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.027 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.028 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.028 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.028 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.028 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.028 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.029 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.029 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.029 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.029 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.029 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.030 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.030 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.032 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.041 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.041 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.041 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.041 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.041 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.042 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.042 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=misc 2024-09-21T11:55:30.042 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.042 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78618 2024-09-21T11:55:30.042 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test misc on pid 78618' 2024-09-21T11:55:30.042 INFO:tasks.workunit.client.0.smithi107.stdout:test misc on pid 78618 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78618 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s pool 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' pool' 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.045 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.047 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78389/exe 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.048 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.049 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo pool 2024-09-21T11:55:30.049 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.049 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=pool 2024-09-21T11:55:30.049 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78646 2024-09-21T11:55:30.049 INFO:tasks.workunit.client.0.smithi107.stdout:test pool on pid 78646 2024-09-21T11:55:30.054 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:30.055 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test pool on pid 78646' 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78646 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.059 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s read_operations 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_ec_io 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' read_operations' 2024-09-21T11:55:30.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78426/exe 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_ec_io.log 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.061 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.062 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo read_operations 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.063 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ ec_io: /' 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.064 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_list 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ list: /' 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_list.log 2024-09-21T11:55:30.068 INFO:tasks.workunit.client.0.smithi107.stdout:test read_operations on pid 78695 2024-09-21T11:55:30.073 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=read_operations 2024-09-21T11:55:30.073 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.073 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78695 2024-09-21T11:55:30.073 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test read_operations on pid 78695' 2024-09-21T11:55:30.073 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78695 2024-09-21T11:55:30.073 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.074 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.075 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.076 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78535/exe 2024-09-21T11:55:30.077 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.078 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.080 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.080 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.081 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.081 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.081 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.081 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.081 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s snapshots 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' snapshots' 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.082 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo snapshots 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_ec_list 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ ec_list: /' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.083 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_ec_list.log 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.084 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.085 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.086 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.086 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.086 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.086 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.086 INFO:tasks.workunit.client.0.smithi107.stdout:test snapshots on pid 78725 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=snapshots 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78725 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test snapshots on pid 78725' 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78725 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.089 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.090 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s watch_notify 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' watch_notify' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo watch_notify 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78646/exe 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.097 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.098 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78618/exe 2024-09-21T11:55:30.098 INFO:tasks.workunit.client.0.smithi107.stdout:test watch_notify on pid 78742 2024-09-21T11:55:30.100 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=watch_notify 2024-09-21T11:55:30.100 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78742 2024-09-21T11:55:30.100 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test watch_notify on pid 78742' 2024-09-21T11:55:30.100 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78742 2024-09-21T11:55:30.100 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.101 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s write_operations 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_pool 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' write_operations' 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.102 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_pool.log 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_misc 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.103 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ misc: /' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.107 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.107 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ pool: /' 2024-09-21T11:55:30.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo write_operations 2024-09-21T11:55:30.107 INFO:tasks.workunit.client.0.smithi107.stdout:test write_operations on pid 78763 2024-09-21T11:55:30.114 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=write_operations 2024-09-21T11:55:30.114 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78763 2024-09-21T11:55:30.114 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test write_operations on pid 78763' 2024-09-21T11:55:30.114 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=78763 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+ ret=0 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77533 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77533 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_misc.log 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-21T11:55:30.122 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.123 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.124 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.125 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.126 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-21T11:55:30.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78725/exe 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.130 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.131 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ snapshots: /' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_snapshots.log 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.132 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-21T11:55:30.133 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_snapshots 2024-09-21T11:55:30.134 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78742/exe 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78695/exe 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ watch_notify: /' 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-21T11:55:30.136 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-21T11:55:30.137 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_read_operations 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_watch_notify 2024-09-21T11:55:30.138 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_read_operations.log 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ read_operations: /' 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/78763/exe 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_neorados_write_operations.log 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_neorados_write_operations 2024-09-21T11:55:30.139 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ write_operations: /' 2024-09-21T11:55:30.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:30.979 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] Global test environment set-up. 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (0 ms) 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (1 ms) 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-21T11:55:30.980 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (13 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (2 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-09-21T11:55:30.981 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] 12 tests from AsioRados (25 ms total) 2024-09-21T11:55:30.982 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: 2024-09-21T11:55:30.982 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] Global test environment tear-down 2024-09-21T11:55:30.982 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (1289 ms total) 2024-09-21T11:55:30.982 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] Global test environment set-up. 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjects (199 ms) 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-21T11:55:31.195 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (20 ms) 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo2,foo3 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo2 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo3 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo4,foo5 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo4 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo5 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo6,foo7 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo7 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo6 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo4 2024-09-21T11:55:31.196 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo5 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo7 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo6 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo1 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo1 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo2 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo3 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (60 ms) 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 1 0 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 10 0 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 13 0 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 7 0 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 14 0 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 0 0 2024-09-21T11:55:31.197 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 15 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 11 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 5 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 8 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 6 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 3 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 4 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 12 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 9 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2 0 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (56 ms) 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: x cursor=MIN 2024-09-21T11:55:31.198 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=1 cursor=13:02547ec2:::1:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=10 cursor=13:52ea6a34:::10:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=13 cursor=13:566253c9:::13:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=7 cursor=13:5c6b0b28:::7:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=14 cursor=13:62a1935d:::14:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=0 cursor=13:6cac518f:::0:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=15 cursor=13:863748b0:::15:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=11 cursor=13:89d3ae78:::11:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=5 cursor=13:b29083e3:::5:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=8 cursor=13:bd63b0f1:::8:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=6 cursor=13:c4fdafeb:::6:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=3 cursor=13:cfc208b3:::3:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=4 cursor=13:d83876eb:::4:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=12 cursor=13:de5d7c5f:::12:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=9 cursor=13:e960b815:::9:head 2024-09-21T11:55:31.199 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=2 cursor=13:f905c69b:::2:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=1 cursor=13:02547ec2:::1:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=10 cursor=13:52ea6a34:::10:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=13 cursor=13:566253c9:::13:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:566253c9:::13:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=7 cursor=13:5c6b0b28:::7:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-09-21T11:55:31.200 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=14 cursor=13:62a1935d:::14:head 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=0 cursor=13:6cac518f:::0:head 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-09-21T11:55:31.251 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=15 cursor=13:863748b0:::15:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:863748b0:::15:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=11 cursor=13:89d3ae78:::11:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=5 cursor=13:b29083e3:::5:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=8 cursor=13:bd63b0f1:::8:head 2024-09-21T11:55:31.252 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=6 cursor=13:c4fdafeb:::6:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=3 cursor=13:cfc208b3:::3:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=4 cursor=13:d83876eb:::4:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=12 cursor=13:de5d7c5f:::12:head 2024-09-21T11:55:31.253 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=9 cursor=13:e960b815:::9:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:e960b815:::9:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=2 cursor=13:f905c69b:::2:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:52ea6a34:::10:head expected=13:52ea6a34:::10:head 2024-09-21T11:55:31.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=10 expected=10 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:f905c69b:::2:head expected=13:f905c69b:::2:head 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=2 expected=2 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:e960b815:::9:head 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:e960b815:::9:head expected=13:e960b815:::9:head 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=9 expected=9 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:c4fdafeb:::6:head expected=13:c4fdafeb:::6:head 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-09-21T11:55:31.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=6 expected=6 2024-09-21T11:55:31.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-09-21T11:55:31.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:bd63b0f1:::8:head expected=13:bd63b0f1:::8:head 2024-09-21T11:55:31.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-09-21T11:55:31.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=8 expected=8 2024-09-21T11:55:31.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-09-21T11:55:31.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:b29083e3:::5:head expected=13:b29083e3:::5:head 2024-09-21T11:55:31.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-09-21T11:55:31.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=5 expected=5 2024-09-21T11:55:31.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-09-21T11:55:31.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:89d3ae78:::11:head expected=13:89d3ae78:::11:head 2024-09-21T11:55:31.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-09-21T11:55:31.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=11 expected=11 2024-09-21T11:55:31.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-09-21T11:55:31.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:02547ec2:::1:head expected=13:02547ec2:::1:head 2024-09-21T11:55:31.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-09-21T11:55:31.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: osdmap e67: 8 total, 8 up, 8 in 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1302293194' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/632640014' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16283010' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/672020936' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3529830612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/255694980' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: osdmap e67: 8 total, 8 up, 8 in 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1302293194' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/632640014' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16283010' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/672020936' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3529830612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/255694980' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1427103447' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1427103447' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-21T11:55:31.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: pgmap v46: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:55:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: osdmap e67: 8 total, 8 up, 8 in 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1302293194' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/632640014' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16283010' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/672020936' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3529830612' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/255694980' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1427103447' entity='client.admin' cmd={"prefix":"quorum_status"} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:31.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:32.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: osdmap e68: 8 total, 8 up, 8 in 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3914720221' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/479788686' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2337994347' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2739811165' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1971489283' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4179157926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3752469748' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/856081801' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4179722660' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2651794622' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1930747595' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.204 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"}]': finished 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: osdmap e69: 8 total, 8 up, 8 in 2024-09-21T11:55:32.206 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: osdmap e68: 8 total, 8 up, 8 in 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3914720221' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/479788686' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-77495-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-77544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-77431-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-77474-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-77452-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-77529-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-77641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi107-78652-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi107-78715-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: osdmap e68: 8 total, 8 up, 8 in 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3914720221' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/479788686' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2337994347' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2739811165' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1971489283' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4179157926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3752469748' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/856081801' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4179722660' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2651794622' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1930747595' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:32.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2337994347' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2739811165' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1971489283' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4179157926' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3752469748' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/856081801' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4179722660' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2651794622' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1930747595' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:55:32.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"}]': finished 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: osdmap e69: 8 total, 8 up, 8 in 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-77942-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78047-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-78241-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-77822-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-77755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-77906-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-77776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-77882-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78050-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"}]': finished 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: osdmap e69: 8 total, 8 up, 8 in 2024-09-21T11:55:32.264 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:55:32.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=1 list_parallel: process_1_[78394]: starting. 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[78394]: creating pool ceph_test_rados_list_parallel.smithi107-78271 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[78394]: created object 0... 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[78394]: created object 25... 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[78394]: created object 49... 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[78394]: finishing. 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[78394]: shutting down. 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[78396]: starting. 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[78396]: listing objects. 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[78396]: listed object 0... 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[78396]: listed object 25... 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[78396]: saw 50 objects 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[78396]: shutting down. 2024-09-21T11:55:32.626 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: starting. 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: creating pool ceph_test_rados_list_parallel.smithi107-78271 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: created object 0... 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: created object 25... 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: created object 49... 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: finishing. 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[79106]: shutting down. 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[79107]: starting. 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[79107]: listing objects. 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[79107]: listed object 0... 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[79107]: listed object 25... 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[79107]: saw 42 objects 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[79107]: shutting down. 2024-09-21T11:55:32.627 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[79108]: starting. 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[79108]: removed 25 objects... 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[79108]: removed half of the objects 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[79108]: removed 50 objects... 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[79108]: removed 50 objects 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[79108]: shutting down. 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: starting. 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: creating pool ceph_test_rados_list_parallel.smithi107-78271 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: created object 0... 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: created object 25... 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: created object 49... 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: finishing. 2024-09-21T11:55:32.628 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[79154]: shutting down. 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: starting. 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: listing objects. 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: listed object 0... 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: listed object 25... 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: listed object 50... 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: saw 56 objects 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[79155]: shutting down. 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[79156]: starting. 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[79156]: added 25 objects... 2024-09-21T11:55:32.629 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[79156]: added half of the objects 2024-09-21T11:55:32.630 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[79156]: added 50 objects... 2024-09-21T11:55:32.630 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[79156]: added 50 objects 2024-09-21T11:55:32.630 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[79156]: shutting down. 2024-09-21T11:55:32.630 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.630 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:55:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:32.809 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ********************** handler_error: Running main() from gmock_main.cc 2024-09-21T11:55:32.809 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-09-21T11:55:32.809 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [----------] Global test environment set-up. 2024-09-21T11:55:32.809 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-09-21T11:55:32.809 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-09-21T11:55:32.809 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (2768 ms) 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [----------] 1 test from neocls_handler_error (2768 ms total) 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [----------] Global test environment tear-down 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [==========] 1 test from 1 test suite ran. (2768 ms total) 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: handler_error: [ PASSED ] 1 test. 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: Running main() from gmock_main.cc 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [==========] Running 1 test from 1 test suite. 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [----------] Global test environment set-up. 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [----------] 1 test from NeoRadosCls 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [ OK ] NeoRadosCls.DNE (2833 ms) 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [----------] 1 test from NeoRadosCls (2833 ms total) 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: 2024-09-21T11:55:32.810 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [----------] Global test environment tear-down 2024-09-21T11:55:32.811 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [==========] 1 test from 1 test suite ran. (2834 ms total) 2024-09-21T11:55:32.811 INFO:tasks.workunit.client.0.smithi107.stdout: cls: [ PASSED ] 1 test. 2024-09-21T11:55:32.824 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-09-21T11:55:32.824 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-21T11:55:32.824 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-21T11:55:32.824 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-21T11:55:32.824 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (57 ms) 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (62 ms) 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2833 ms) 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (2952 ms total) 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (2952 ms total) 2024-09-21T11:55:32.825 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-21T11:55:32.828 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-09-21T11:55:32.828 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-09-21T11:55:32.828 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-21T11:55:32.828 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-09-21T11:55:32.828 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (242 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (37 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (27 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (16 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (12 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (7 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (7 ms) 2024-09-21T11:55:32.829 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (10 ms) 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (12 ms) 2024-09-21T11:55:32.830 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (13 ms) 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (419 ms total) 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2870 ms total) 2024-09-21T11:55:32.831 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (71 ms) 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (74 ms) 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2794 ms) 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-21T11:55:32.887 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.812186+0000 mon.a [WRN] Health check failed: 27 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.871027+0000 mon.b [INF] from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.872182+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.875930+0000 mon.b [INF] from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.877065+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.878115+0000 mon.b [INF] from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.878877+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.880268+0000 mon.b [INF] from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.881905+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.882983+0000 mon.b [INF] from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.883875+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.887602+0000 mon.b [INF] from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:31.906996+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:32.888 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.786225+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.786328+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.786435+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.786515+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.786595+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.786683+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.794749+0000 mon.b [INF] from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.795825+0000 mon.b [INF] from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.805974+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.806846+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.809117+0000 mon.b [INF] from='client.? 172.21.15.107:0/3480052684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.812260+0000 mon.a [INF] from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.812835+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:32.889 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.828112+0000 mon.c [INF] from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11********* 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: starting. 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: creating pool ceph_test_rados_list_parallel.smithi107-78271 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: created object 0... 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: created object 25... 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: created object 49... 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: finishing. 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[79202]: shutting down. 2024-09-21T11:55:33.068 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: starting. 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: listing objects. 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: listed object 0... 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: listed object 25... 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: listed object 50... 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: listed object 75... 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: saw 97 objects 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[79203]: shutting down. 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.069 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[79206]: starting. 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[79206]: removed 25 objects... 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[79206]: removed half of the objects 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[79206]: removed 50 objects... 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[79206]: removed 50 objects 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[79206]: shutting down. 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[79204]: starting. 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[79204]: added 25 objects... 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[79204]: added half of the objects 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[79204]: added 50 objects... 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[79204]: added 50 objects 2024-09-21T11:55:33.070 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[79204]: shutting down. 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[79205]: starting. 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[79205]: added 25 objects... 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[79205]: added half of the objects 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[79205]: added 50 objects... 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[79205]: added 50 objects 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[79205]: shutting down. 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.071 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: starting. 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: creating pool ceph_test_rados_list_parallel.smithi107-78271 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: created object 0... 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: created object 25... 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: created object 49... 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: finishing. 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[79282]: shutting down. 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: starting. 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listing objects. 2024-09-21T11:55:33.072 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listed object 0... 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: pgmap v49: 1220 pgs: 1088 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: Health check failed: 27 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: pgmap v49: 1220 pgs: 1088 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: Health check failed: 27 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:33.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: osdmap e70: 8 total, 8 up, 8 in 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3480052684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: osdmap e70: 8 total, 8 up, 8 in 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3480052684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: pgmap v49: 1220 pgs: 1088 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: Health check failed: 27 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi107-78715-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-77958-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi107-78652-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-77474-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: osdmap e70: 8 total, 8 up, 8 in 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3480052684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:33.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.864 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listed object 25... 2024-09-21T11:55:33.864 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listed object 50... 2024-09-21T11:55:33.864 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listed object 75... 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listed object 100... 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: listed object 125... 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: saw 150 objects 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[79283]: shutting down. 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[79284]: starting. 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[79284]: added 25 objects... 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[79284]: added half of the objects 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[79284]: added 50 objects... 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[79284]: added 50 objects 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[79284]: shutting down. 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.865 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.866 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.866 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.866 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-21T11:55:33.866 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******* SUCCESS ********** 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout::55:32.833205+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.878342+0000 mon.b [INF] from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.879410+0000 client.admin [INF] onexx 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.879905+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.921780+0000 mon.b [INF] from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.963426+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.970455+0000 mon.b [INF] from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.982082+0000 mon.a [INF] from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.982316+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.986429+0000 mon.b [INF] from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.986455+0000 mon.a [INF] from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:33.905 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.987772+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.906 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:32.989717+0000 mon.a [INF] from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:33.906 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:33.821999+0000 mon.a [INF] from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:33.906 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:33.822102+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:33.906 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:33.822194+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:33.906 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:33.822272+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: onexx 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: osdmap e71: 8 total, 8 up, 8 in 2024-09-21T11:55:34.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: onexx 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:34.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: osdmap e71: 8 total, 8 up, 8 in 2024-09-21T11:55:34.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: onexx 2024-09-21T11:55:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-21T11:55:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-77431-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:34.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: osdmap e71: 8 total, 8 up, 8 in 2024-09-21T11:55:34.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: pgmap v52: 700 pgs: 216 creating+peering, 484 active+clean; 609 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 133 KiB/s wr, 509 op/s 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2036616296' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2598755718' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: twoxx 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-77544-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: pgmap v52: 700 pgs: 216 creating+peering, 484 active+clean; 609 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 133 KiB/s wr, 509 op/s 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2036616296' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2598755718' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: twoxx 2024-09-21T11:55:35.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-77544-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-21T11:55:35.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2598755718' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-77544-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:35.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-21T11:55:35.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2598755718' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-77544-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: osdmap e72: 8 total, 8 up, 8 in 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:35.261 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: osdmap e72: 8 total, 8 up, 8 in 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:35.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: pgmap v52: 700 pgs: 216 creating+peering, 484 active+clean; 609 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 133 KiB/s wr, 509 op/s 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2036616296' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2598755718' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: twoxx 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["twoxx"]} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-77544-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-21T11:55:35.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-77474-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2598755718' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-77544-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: osdmap e72: 8 total, 8 up, 8 in 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:35.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: api_c delete_pools_parallel: process_1_[78374]: starting. 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[78374]: creating pool ceph_test_rados_delete_pools_parallel.smithi107-78300 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[78374]: created object 0... 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[78374]: created object 25... 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[78374]: created object 49... 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[78374]: finishing. 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[78374]: shutting down. 2024-09-21T11:55:35.869 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_2_[78376]: starting. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_2_[78376]: deleting pool ceph_test_rados_delete_pools_parallel.smithi107-78300 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_2_[78376]: shutting down. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: starting. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: creating pool ceph_test_rados_delete_pools_parallel.smithi107-78300 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: created object 0... 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: created object 25... 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: created object 49... 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: finishing. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[79334]: shutting down. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[79337]: starting. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[79337]: listing objects. 2024-09-21T11:55:35.870 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[79337]: listed object 0... 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[79337]: listed object 25... 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[79337]: saw 50 objects 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[79337]: shutting down. 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_4_[79335]: starting. 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_4_[79335]: deleting pool ceph_test_rados_delete_pools_parallel.smithi107-78300 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_4_[79335]: shutting down. 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-21T11:55:35.871 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-21T11:55:35.885 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: starting. 2024-09-21T11:55:35.885 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: creating pool ceph_test_rados_open_pools_parallel.smithi107-78296 2024-09-21T11:55:35.885 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: created object 0... 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: created object 25... 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: created object 49... 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: finishing. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[78371]: shutting down. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[78372]: starting. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[78372]: rados_pool_create. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[78372]: rados_ioctx_create. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[78372]: shutting down. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: starting. 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: creating pool ceph_test_rados_open_pools_parallel.smithi107-78296 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: created object 0... 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: created object 25... 2024-09-21T11:55:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: created object 49... 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: finishing. 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[79333]: shutting down. 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[79336]: starting. 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[79336]: rados_pool_create. 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[79336]: rados_ioctx_create. 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[79336]: shutting down. 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-21T11:55:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-21T11:55:36.041 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: Running main() from gmock_main.cc 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [==========] Running 40 tests from 2 test suites. 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: seed 77495 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (139 ms) 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (15 ms) 2024-09-21T11:55:36.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (17 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (225 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (4 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (3 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (4 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (2 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-21T11:55:36.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (2 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (2 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (7 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (2 ms) 2024-09-21T11:55:36.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CrcZeroWrite 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CrcZeroWrite (3 ms) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (5 ms) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 22 tests from LibRadosIoPP (461 ms total) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1180 ms) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (17 ms) 2024-09-21T11:55:36.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-21T11:55:36.046 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-09-21T11:55:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-21T11:55:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-21T11:55:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-21T11:55:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-21T11:55:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.0 deep-scrub starts 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.0 deep-scrub ok 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.5 deep-scrub starts 2024-09-21T11:55:36.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.5 deep-scrub ok 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.2 deep-scrub starts 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.2 deep-scrub ok 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1430405280' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.9 deep-scrub starts 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: 17.9 deep-scrub ok 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:36.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-21T11:55:36.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.0 deep-scrub starts 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.0 deep-scrub ok 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.5 deep-scrub starts 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.5 deep-scrub ok 2024-09-21T11:55:36.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.2 deep-scrub starts 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.2 deep-scrub ok 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1430405280' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.9 deep-scrub starts 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: 17.9 deep-scrub ok 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:36.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:36.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.261 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.261 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.261 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: osdmap e73: 8 total, 8 up, 8 in 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1415139923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.262 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: osdmap e73: 8 total, 8 up, 8 in 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.263 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.264 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.264 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1415139923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.264 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.264 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.264 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:36.264 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.0 deep-scrub starts 2024-09-21T11:55:36.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.0 deep-scrub ok 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.5 deep-scrub starts 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.5 deep-scrub ok 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.2 deep-scrub starts 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.2 deep-scrub ok 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1430405280' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.9 deep-scrub starts 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: 17.9 deep-scrub ok 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:36.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: osdmap e73: 8 total, 8 up, 8 in 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1415139923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:36.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.868 INFO:tasks.workunit.client.0.smithi107.stdout: a cmd: Running main() from gmock_main.cc 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [----------] Global test environment set-up. 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (1775 ms) 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2064 ms) 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (2985 ms) 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [----------] 3 tests from NeoRadosCmd (6824 ms total) 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [----------] Global test environment tear-down 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [==========] 3 tests from 1 test suite ran. (6824 ms total) 2024-09-21T11:55:36.869 INFO:tasks.workunit.client.0.smithi107.stdout: cmd: [ PASSED ] 3 tests. 2024-09-21T11:55:36.920 INFO:tasks.workunit.client.0.smithi107.stdout:md: got: 2024-09-21T11:55:33.822345+0000 mon.a [INF] from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-77882-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:36.920 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:33.826906+0000 mon.b [INF] from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:33.848437+0000 mon.b [INF] from='client.? 172.21.15.107:0/2650078737' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi107-78652-1"} : dispatch 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:34.904864+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833311+0000 mon.a [INF] from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-77882-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833396+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-77942-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833484+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi107-78652-1"}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833547+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-77452-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833608+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833674+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.833723+0000 mon.a [INF] from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.839073+0000 mon.b [INF] from='client.? 172.21.15.107:0/3180028057' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.839603+0000 mon.b [INF] from='client.? 172.21.15.107:0/771329264' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.921 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.851504+0000 mon.c [INF] from='client.? 172.21.15.107:0/2087162685' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.922 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.864318+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"} : dispatch 2024-09-21T11:55:36.958 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.864668+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.865008+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.875092+0000 mon.c [INF] from='client.? 172.21.15.107:0/1415139923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.877116+0000 mon.b [INF] from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.887922+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.888171+0000 mon.a [INF] from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:35.888581+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.093938+0000 mon.b [INF] from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.094318+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.838518+0000 mon.a [INF] from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.838619+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.838699+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:36.959 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.838759+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-21T11:55:36.960 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.838850+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:36.960 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.838949+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:36.960 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.839007+0000 mon.a [INF] from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.1 deep-scrub starts 2024-09-21T11:55:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: pgmap v55: 884 pgs: 464 unknown, 64 creating+peering, 356 active+clean; 586 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 21 KiB/s wr, 354 op/s 2024-09-21T11:55:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.1 deep-scrub ok 2024-09-21T11:55:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.8 deep-scrub starts 2024-09-21T11:55:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.8 deep-scrub ok 2024-09-21T11:55:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.3 deep-scrub starts 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.3 deep-scrub ok 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.6 deep-scrub starts 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.6 deep-scrub ok 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.4 deep-scrub starts 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: 17.4 deep-scrub ok 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-21T11:55:37.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: osdmap e74: 8 total, 8 up, 8 in 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.1 deep-scrub starts 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: pgmap v55: 884 pgs: 464 unknown, 64 creating+peering, 356 active+clean; 586 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 21 KiB/s wr, 354 op/s 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.1 deep-scrub ok 2024-09-21T11:55:37.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.8 deep-scrub starts 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.8 deep-scrub ok 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.3 deep-scrub starts 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.3 deep-scrub ok 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.6 deep-scrub starts 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.6 deep-scrub ok 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.4 deep-scrub starts 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: 17.4 deep-scrub ok 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-21T11:55:37.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: osdmap e74: 8 total, 8 up, 8 in 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/354328586' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3429039264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: threexx 2024-09-21T11:55:37.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/354328586' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3429039264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: threexx 2024-09-21T11:55:37.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.1 deep-scrub starts 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: pgmap v55: 884 pgs: 464 unknown, 64 creating+peering, 356 active+clean; 586 KiB data, 263 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 21 KiB/s wr, 354 op/s 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.1 deep-scrub ok 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.8 deep-scrub starts 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.8 deep-scrub ok 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.3 deep-scrub starts 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.3 deep-scrub ok 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.6 deep-scrub starts 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.6 deep-scrub ok 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T11:55:37.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.4 deep-scrub starts 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: 17.4 deep-scrub ok 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-77544-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-77641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-77958-1"}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-77452-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi107-78715-1"}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-77431-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:37.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: osdmap e74: 8 total, 8 up, 8 in 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/354328586' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3429039264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:37.303 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: threexx 2024-09-21T11:55:37.304 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: fourxx 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3641036940' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: 17.7 deep-scrub starts 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: 17.7 deep-scrub ok 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: osdmap e75: 8 total, 8 up, 8 in 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-4-cache", "mode": "writeback"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi107-78715-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: fourxx 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3641036940' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: 17.7 deep-scrub starts 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: 17.7 deep-scrub ok 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:38.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: osdmap e75: 8 total, 8 up, 8 in 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-4-cache", "mode": "writeback"} : dispatch 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi107-78715-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:38.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:38.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-21T11:55:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-21T11:55:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: fourxx 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["fourxx"]} : dispatch 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3641036940' entity='client.admin' cmd={"prefix":"status"} : dispatch 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: 17.7 deep-scrub starts 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: 17.7 deep-scrub ok 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:38.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: osdmap e75: 8 total, 8 up, 8 in 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-4-cache", "mode": "writeback"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsNSsmithi107-78715-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:38.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_c api_io: Running main() from gmock_main.cc 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] Global test environment set-up. 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (181 ms) 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-21T11:55:38.856 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (28 ms) 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (9 ms) 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.Checksum (2 ms) 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (2 ms) 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-21T11:55:38.857 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (4 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.TruncTest (2 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (3 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.RmXattr (7 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.XattrIter (3 ms) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 14 tests from LibRadosIo (251 ms total) 2024-09-21T11:55:38.858 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1179 ms) 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (10 ms) 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (9 ms) 2024-09-21T11:55:38.859 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (8 ms) 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (2 ms) 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (8 ms) 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (3 ms) 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1231 ms total) 2024-09-21T11:55:38.860 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: 2024-09-21T11:55:38.861 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] Global test environment tear-down 2024-09-21T11:55:38.861 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [==========] 24 tests from 2 test suites ran. (9212 ms total) 2024-09-21T11:55:38.861 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ PASSED ] 24 tests. 2024-09-21T11:55:38.871 INFO:tasks.workunit.client.0.smithi107.stdout: list: Running main() from gmock_main.cc 2024-09-21T11:55:38.871 INFO:tasks.workunit.client.0.smithi107.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [----------] Global test environment set-up. 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [----------] 3 tests from NeoradosList 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ OK ] NeoradosList.ListObjects (2738 ms) 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3069 ms) 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ OK ] NeoradosList.ListObjectsMany (2986 ms) 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [----------] 3 tests from NeoradosList (8793 ms total) 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [----------] Global test environment tear-down 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [==========] 3 tests from 1 test suite ran. (8793 ms total) 2024-09-21T11:55:38.872 INFO:tasks.workunit.client.0.smithi107.stdout: list: [ PASSED ] 3 tests. 2024-09-21T11:55:38.994 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: pgmap v58: 908 pgs: 488 unknown, 64 creating+peering, 356 active+clean; 586 KiB data, 263 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-4-cache", "mode": "writeback"}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: osdmap e76: 8 total, 8 up, 8 in 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2179798597' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-77431-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: pgmap v58: 908 pgs: 488 unknown, 64 creating+peering, 356 active+clean; 586 KiB data, 263 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-4-cache", "mode": "writeback"}]': finished 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:38.996 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: osdmap e76: 8 total, 8 up, 8 in 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:38.997 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2179798597' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-77431-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: pgmap v58: 908 pgs: 488 unknown, 64 creating+peering, 356 active+clean; 586 KiB data, 263 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:55:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-21T11:55:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-4-cache", "mode": "writeback"}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-77474-16"}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/345030014' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3591227771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: osdmap e76: 8 total, 8 up, 8 in 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"} : dispatch 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"} : dispatch 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2179798597' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-77431-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:39.869 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: Running main() from gmock_main.cc 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] Global test environment set-up. 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.Stat (306 ms) 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (27 ms) 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.StatNS (17 ms) 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-21T11:55:39.870 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 5 tests from LibRadosStat (352 ms total) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1075 ms) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (7 ms) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (1 ms) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1084 ms total) 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: 2024-09-21T11:55:39.871 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] Global test environment tear-down 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (10075 ms total) 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ PASSED ] 9 tests. 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: Running main() from gmock_main.cc 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [----------] Global test environment set-up. 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: handle_notify cookie 94634096410304 notify_id 296352743424 notifier_gid 24839 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (2688 ms) 2024-09-21T11:55:39.872 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: Trying... 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: handle_notify cookie 94634109115056 notify_id 309237645313 notifier_gid 24958 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: Waiting for 3.000000000s 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: Timed out. 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: Flushing... 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: Flushed... 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7052 ms) 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (9740 ms total) 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [----------] Global test environment tear-down 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (9740 ms total) 2024-09-21T11:55:39.873 INFO:tasks.workunit.client.0.smithi107.stdout: watch_notify: [ PASSED ] 2 tests. 2024-09-21T11:55:39.876 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: seed 77942 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (301 ms) 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (26 ms) 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (6 ms) 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (13 ms) 2024-09-21T11:55:39.877 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (346 ms total) 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1125 ms) 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2 ms) 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (8 ms) 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1136 ms total) 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-21T11:55:39.878 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (10009 ms total) 2024-09-21T11:55:39.879 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-21T11:55:39.983 INFO:tasks.workunit.client.0.smithi107.stdout:md: got: 2024-09-21T11:55:36.839067+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi107-78652-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:39.983 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.839134+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "LibRadosIoECPP_smithi107-77495-24", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T11:55:39.983 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.845599+0000 mon.b [INF] from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:39.983 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.849682+0000 mon.b [INF] from='client.? 172.21.15.107:0/354328586' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:39.983 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.850187+0000 mon.b [INF] from='client.? 172.21.15.107:0/3429039264' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.851088+0000 mon.b [INF] from='client.? 172.21.15.107:0/1056558387' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.852719+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReadOpsmithi107-78652-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.853436+0000 mon.a [INF] from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-4", "overlaypool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.861394+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.862050+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.862571+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-77474-16"} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.895754+0000 mon.a [INF] from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi107-78715-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.918726+0000 mon.b [INF] from='client.? 172.21.15.107:0/401919040' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.920863+0000 client.admin [INF] threexx 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-21T11:55:36.928252+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7173 ms) 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (10112 ms total) 2024-09-21T11:55:39.984 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: 2024-09-21T11:55:39.985 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-21T11:55:39.985 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (10112 ms total) 2024-09-21T11:55:39.985 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-21T11:55:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4} : dispatch 2024-09-21T11:55:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-4"} : dispatch 2024-09-21T11:55:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4} : dispatch 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi107-78715-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2179798597' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-77431-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-4"}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4}]': finished 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: osdmap e77: 8 total, 8 up, 8 in 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1190181797' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3610655064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-4"} : dispatch 2024-09-21T11:55:40.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4} : dispatch 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi107-78715-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2179798597' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-77431-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-4"}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4}]': finished 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: osdmap e77: 8 total, 8 up, 8 in 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1190181797' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3610655064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:40.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:40.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4} : dispatch 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-4"} : dispatch 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4} : dispatch 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi107-78715-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/183498945' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-77882-7"}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-77942-7"}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2179798597' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-77431-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-4"}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77958-6", "pg_num": 4}]': finished 2024-09-21T11:55:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: osdmap e77: 8 total, 8 up, 8 in 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1190181797' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3610655064' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: pgmap v61: 764 pgs: 64 creating+peering, 72 unknown, 628 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 200 op/s 2024-09-21T11:55:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:55:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: osdmap e78: 8 total, 8 up, 8 in 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: pgmap v61: 764 pgs: 64 creating+peering, 72 unknown, 628 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 200 op/s 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:55:41.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: osdmap e78: 8 total, 8 up, 8 in 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:41.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:41.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:41.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:41.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:41.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:41.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:41.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: pgmap v61: 764 pgs: 64 creating+peering, 72 unknown, 628 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 200 op/s 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1555078236' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-4", "tierpool": "test-rados-api-smithi107-77755-4-cache"}]': finished 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: osdmap e78: 8 total, 8 up, 8 in 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:41.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:41.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:41.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:41.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:41.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:41.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:41.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: Running main() from gmock_main.cc 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [----------] Global test environment set-up. 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (5770 ms) 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (6403 ms) 2024-09-21T11:55:42.246 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (12173 ms total) 2024-09-21T11:55:42.247 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: 2024-09-21T11:55:42.247 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [----------] Global test environment tear-down 2024-09-21T11:55:42.247 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (12173 ms total) 2024-09-21T11:55:42.247 INFO:tasks.workunit.client.0.smithi107.stdout: ec_io: [ PASSED ] 2 tests. 2024-09-21T11:55:42.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-21T11:55:42.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-21T11:55:42.254 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: seed 77641 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (150 ms) 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (12 ms) 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1242 ms) 2024-09-21T11:55:42.255 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1004 ms) 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3 ms) 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (3 ms) 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (5 ms) 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-21T11:55:42.256 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2422 ms total) 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1125 ms) 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (9 ms) 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1078 ms) 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-21T11:55:42.257 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (3 ms) 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3231 ms total) 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-21T11:55:42.258 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (12545 ms total) 2024-09-21T11:55:42.259 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: Running main() from gmock_main.cc 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] Global test environment set-up. 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (157 ms) 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockShared (8 ms) 2024-09-21T11:55:42.263 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1138 ms) 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1014 ms) 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.Unlock (4 ms) 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (6 ms) 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (2 ms) 2024-09-21T11:55:42.264 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLock (2333 ms total) 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1117 ms) 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (4 ms) 2024-09-21T11:55:42.265 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1100 ms) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3239 ms total) 2024-09-21T11:55:42.266 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: 2024-09-21T11:55:42.267 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] Global test environment tear-down 2024-09-21T11:55:42.267 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (12588 ms total) 2024-09-21T11:55:42.267 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-21T11:55:42.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"}]': finished 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: osdmap e79: 8 total, 8 up, 8 in 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:42.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224136096' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:42.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4206937512' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:55:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"}]': finished 2024-09-21T11:55:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:42.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"} : dispatch 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: osdmap e79: 8 total, 8 up, 8 in 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"} : dispatch 2024-09-21T11:55:42.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224136096' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4206937512' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:42.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "overlaypool": "test-rados-api-smithi107-77958-6"}]': finished 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/532643508' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"} : dispatch 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/915212519' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: osdmap e79: 8 total, 8 up, 8 in 2024-09-21T11:55:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224136096' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4206937512' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:42.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.259 INFO:tasks.workunit.client.0.smithi107.stdout: pool: Running main() from gmock_main.cc 2024-09-21T11:55:43.259 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [----------] Global test environment set-up. 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ OK ] NeoRadosPools.PoolList (1723 ms) 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (2031 ms) 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2013 ms) 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4009 ms) 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2016 ms) 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (1364 ms) 2024-09-21T11:55:43.260 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [----------] 6 tests from NeoRadosPools (13156 ms total) 2024-09-21T11:55:43.261 INFO:tasks.workunit.client.0.smithi107.stdout: pool: 2024-09-21T11:55:43.261 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [----------] Global test environment tear-down 2024-09-21T11:55:43.261 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [==========] 6 tests from 1 test suite ran. (13156 ms total) 2024-09-21T11:55:43.261 INFO:tasks.workunit.client.0.smithi107.stdout: pool: [ PASSED ] 6 tests. 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: pgmap v64: 752 pgs: 32 creating+peering, 268 unknown, 452 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 195 op/s 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: pool 'PoolQuotaPP_smithi107-77452-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"}]': finished 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:43.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: osdmap e80: 8 total, 8 up, 8 in 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2722926436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2394605387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: pgmap v64: 752 pgs: 32 creating+peering, 268 unknown, 452 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 195 op/s 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: pool 'PoolQuotaPP_smithi107-77452-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"}]': finished 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: osdmap e80: 8 total, 8 up, 8 in 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2722926436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2394605387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-21T11:55:43.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: pgmap v64: 752 pgs: 32 creating+peering, 268 unknown, 452 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 195 op/s 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: pool 'PoolQuotaPP_smithi107-77452-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-77495-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2782023494' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-77544-10"}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-77641-10"}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77958-6", "mode": "writeback"}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi107-78652-2"}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-77431-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-77776-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: osdmap e80: 8 total, 8 up, 8 in 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi107-78715-2"} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2722926436' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2394605387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd dump"} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:43.508 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: osdmap e81: 8 total, 8 up, 8 in 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2"} : dispatch 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: pgmap v68: 624 pgs: 32 creating+peering, 268 unknown, 324 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: osdmap e81: 8 total, 8 up, 8 in 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2"} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: pgmap v68: 624 pgs: 32 creating+peering, 268 unknown, 324 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1615116007' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi107-78715-2"}]': finished 2024-09-21T11:55:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: osdmap e81: 8 total, 8 up, 8 in 2024-09-21T11:55:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2"} : dispatch 2024-09-21T11:55:44.553 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:44.553 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: pgmap v68: 624 pgs: 32 creating+peering, 268 unknown, 324 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:44.554 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.555 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:44.555 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1218 ms) 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1222 ms total) 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: 2024-09-21T11:55:45.374 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (152 ms) 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3316 ms) 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (14 ms) 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (4 ms) 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-21T11:55:45.375 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185047968 notify_id 343597383683 notifier_gid 25180 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185044656 notify_id 343597383684 notifier_gid 25180 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185044656 notify_id 343597383682 notifier_gid 25180 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185044656 notify_id 343597383683 notifier_gid 25180 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185044656 notify_id 343597383685 notifier_gid 25180 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4 ms) 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185046784 notify_id 343597383685 notifier_gid 25180 2024-09-21T11:55:45.376 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-09-21T11:55:45.377 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-21T11:55:45.377 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: trying... 2024-09-21T11:55:45.377 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185046784 notify_id 343597383686 notifier_gid 25180 2024-09-21T11:55:45.377 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: timed out 2024-09-21T11:55:45.377 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-21T11:55:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: osdmap e82: 8 total, 8 up, 8 in 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1147418205' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-77431-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:55:45.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:45.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: osdmap e82: 8 total, 8 up, 8 in 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1147418205' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-77431-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:55:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-77611-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi107-78715-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: osdmap e82: 8 total, 8 up, 8 in 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1147418205' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-77431-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:45.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:45.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"dne","key":"key","value":"value"} : dispatch 2024-09-21T11:55:45.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:55:45.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:55:45.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:46.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_watpi_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-21T11:55:46.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (6 ms) 2024-09-21T11:55:46.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-21T11:55:46.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (6 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (11 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (5 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-21T11:55:46.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CrcZeroWrite 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CrcZeroWrite (6156 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (1133 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (9 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (5 ms) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 18 tests from LibRadosIoECPP (8561 ms total) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [==========] 40 tests from 2 test suites ran. (16622 ms total) 2024-09-21T11:55:46.279 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ PASSED ] 40 tests. 2024-09-21T11:55:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1147418205' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-77431-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: osdmap e83: 8 total, 8 up, 8 in 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2926394031' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1231408332' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1639108409' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-77822-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: pgmap v71: 680 pgs: 288 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1639108409' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-77822-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: osdmap e84: 8 total, 8 up, 8 in 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.552 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1147418205' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-77431-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: osdmap e83: 8 total, 8 up, 8 in 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2926394031' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1231408332' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1639108409' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-77822-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: pgmap v71: 680 pgs: 288 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1147418205' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-77431-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: osdmap e83: 8 total, 8 up, 8 in 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2926394031' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1231408332' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1639108409' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-77822-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: pgmap v71: 680 pgs: 288 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi107-78715-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-77906-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/266331' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-77495-24"}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1639108409' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-77822-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: osdmap e84: 8 total, 8 up, 8 in 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:55:46.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1639108409' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-77822-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: osdmap e84: 8 total, 8 up, 8 in 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/31883387' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:46.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:55:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"}]': finished 2024-09-21T11:55:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[30293]: osdmap e85: 8 total, 8 up, 8 in 2024-09-21T11:55:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1029523764' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"}]': finished 2024-09-21T11:55:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[41097]: osdmap e85: 8 total, 8 up, 8 in 2024-09-21T11:55:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1029523764' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-77611-1","app":"app1","key":"key1"}]': finished 2024-09-21T11:55:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:47 smithi193 ceph-mon[33333]: osdmap e85: 8 total, 8 up, 8 in 2024-09-21T11:55:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1029523764' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: pgmap v74: 600 pgs: 208 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-09-21T11:55:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T11:55:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: osdmap e86: 8 total, 8 up, 8 in 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-7"} : dispatch 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/8160218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: pgmap v74: 600 pgs: 208 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T11:55:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: osdmap e86: 8 total, 8 up, 8 in 2024-09-21T11:55:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-7"} : dispatch 2024-09-21T11:55:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/8160218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:48.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: pgmap v74: 600 pgs: 208 unknown, 392 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-77431-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: osdmap e86: 8 total, 8 up, 8 in 2024-09-21T11:55:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-7"} : dispatch 2024-09-21T11:55:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/8160218' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-7"}]': finished 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/8160218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: osdmap e87: 8 total, 8 up, 8 in 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:49.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:49.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:49.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-7"}]': finished 2024-09-21T11:55:49.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/8160218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:49.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:49.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: osdmap e87: 8 total, 8 up, 8 in 2024-09-21T11:55:49.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:49.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-7"}]': finished 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/8160218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-78136-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: osdmap e87: 8 total, 8 up, 8 in 2024-09-21T11:55:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4292841920' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"} : dispatch 2024-09-21T11:55:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify_test_cb 2024-09-21T11:55:50.389 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (341 ms) 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93917784055312 err -107 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1010 ms) 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93917784055312 err -107 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1019 ms) 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678017 cookie 93917785435136 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (8 ms) 2024-09-21T11:55:50.390 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678017 cookie 93917785484016 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (7 ms) 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678016 cookie 93917785494352 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (4 ms) 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678016 cookie 93917785494352 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678016 cookie 93917785497664 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (8 ms) 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 304942678017 cookie 93917785497664 2024-09-21T11:55:50.391 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 309237645314 cookie 93917785497664 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93917785497536 err -107 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24664 notify_id 339302416388 cookie 93917785497536 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93917785497536 err -107 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11413 ms total) 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-21T11:55:50.392 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify_test_cb 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1062 ms) 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1062 ms total) 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (20562 ms total) 2024-09-21T11:55:50.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-21T11:55:50.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: pgmap v77: 616 pgs: 118 creating+peering, 74 unknown, 424 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:55:50.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:55:50.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:50.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:50.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: osdmap e88: 8 total, 8 up, 8 in 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7"} : dispatch 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2076757285' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/30054806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1787654954' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: pgmap v77: 616 pgs: 118 creating+peering, 74 unknown, 424 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: osdmap e88: 8 total, 8 up, 8 in 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7"} : dispatch 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2076757285' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/30054806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1787654954' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: pgmap v77: 616 pgs: 118 creating+peering, 74 unknown, 424 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-77906-12"}]': finished 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:55:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: osdmap e88: 8 total, 8 up, 8 in 2024-09-21T11:55:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7"} : dispatch 2024-09-21T11:55:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2076757285' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/30054806' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1787654954' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:51.326 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: Running main() from gmock_main.cc 2024-09-21T11:55:51.326 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-09-21T11:55:51.326 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [----------] Global test environment set-up. 2024-09-21T11:55:51.326 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6762 ms) 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (6396 ms) 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (8076 ms) 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [----------] 3 tests from NeoradosECList (21234 ms total) 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [----------] Global test environment tear-down 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (21235 ms total) 2024-09-21T11:55:51.327 INFO:tasks.workunit.client.0.smithi107.stdout: ec_list: [ PASSED ] 3 tests. 2024-09-21T11:55:51.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:51.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:51.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7"}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1787654954' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[30293]: osdmap e89: 8 total, 8 up, 8 in 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7"}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1787654954' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:51.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:51 smithi107 ceph-mon[41097]: osdmap e89: 8 total, 8 up, 8 in 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2545798858' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"} : dispatch 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-7"}]': finished 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1787654954' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-77431-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi107-78715-3"}]': finished 2024-09-21T11:55:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:51 smithi193 ceph-mon[33333]: osdmap e89: 8 total, 8 up, 8 in 2024-09-21T11:55:52.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-mon[30293]: pgmap v79: 648 pgs: 57 creating+peering, 167 unknown, 424 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:55:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-mon[30293]: osdmap e90: 8 total, 8 up, 8 in 2024-09-21T11:55:52.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:55:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-mon[41097]: pgmap v79: 648 pgs: 57 creating+peering, 167 unknown, 424 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:55:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:52 smithi107 ceph-mon[41097]: osdmap e90: 8 total, 8 up, 8 in 2024-09-21T11:55:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:52 smithi193 ceph-mon[33333]: pgmap v79: 648 pgs: 57 creating+peering, 167 unknown, 424 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:55:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:52 smithi193 ceph-mon[33333]: osdmap e90: 8 total, 8 up, 8 in 2024-09-21T11:55:53.305 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: Running main() from gmock_main.cc 2024-09-21T11:55:53.305 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-09-21T11:55:53.305 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [----------] Global test environment set-up. 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (2672 ms) 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3068 ms) 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (2999 ms) 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3013 ms) 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (2378 ms) 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3011 ms) 2024-09-21T11:55:53.306 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-09-21T11:55:53.307 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (6008 ms) 2024-09-21T11:55:53.307 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (23149 ms total) 2024-09-21T11:55:53.307 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: 2024-09-21T11:55:53.307 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [----------] Global test environment tear-down 2024-09-21T11:55:53.307 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (23149 ms total) 2024-09-21T11:55:53.307 INFO:tasks.workunit.client.0.smithi107.stdout: write_operations: [ PASSED ] 7 tests. 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (2919 ms) 2024-09-21T11:55:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3048 ms) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (2996 ms) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3017 ms) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (2387 ms) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3011 ms) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (6040 ms) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (23418 ms total) 2024-09-21T11:55:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: 2024-09-21T11:55:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-21T11:55:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (23418 ms total) 2024-09-21T11:55:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-21T11:55:54.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: osdmap e91: 8 total, 8 up, 8 in 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1924952660' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1992895616' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: osdmap e91: 8 total, 8 up, 8 in 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1924952660' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1992895616' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: osdmap e91: 8 total, 8 up, 8 in 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1924952660' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1992895616' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: pgmap v83: 584 pgs: 192 unknown, 392 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:55:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: osdmap e92: 8 total, 8 up, 8 in 2024-09-21T11:55:55.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:55.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"} : dispatch 2024-09-21T11:55:55.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"} : dispatch 2024-09-21T11:55:55.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: pgmap v83: 584 pgs: 192 unknown, 392 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:55:55.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:55.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:55.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: osdmap e92: 8 total, 8 up, 8 in 2024-09-21T11:55:55.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:55.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"} : dispatch 2024-09-21T11:55:55.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"} : dispatch 2024-09-21T11:55:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: pgmap v83: 584 pgs: 192 unknown, 392 active+clean; 145 MiB data, 805 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:55:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-77431-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-77452-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: osdmap e92: 8 total, 8 up, 8 in 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"} : dispatch 2024-09-21T11:55:55.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"} : dispatch 2024-09-21T11:55:56.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"}]': finished 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: osdmap e93: 8 total, 8 up, 8 in 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-9"} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1087361647' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:56.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"}]': finished 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: osdmap e93: 8 total, 8 up, 8 in 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-9"} : dispatch 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1087361647' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:56.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:55:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4"}]': finished 2024-09-21T11:55:56.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: osdmap e93: 8 total, 8 up, 8 in 2024-09-21T11:55:56.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-9"} : dispatch 2024-09-21T11:55:56.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:56.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"} : dispatch 2024-09-21T11:55:56.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1087361647' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:56.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: pgmap v86: 552 pgs: 32 creating+peering, 1 active+clean+snaptrim, 128 unknown, 391 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-9"}]': finished 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"}]': finished 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: osdmap e94: 8 total, 8 up, 8 in 2024-09-21T11:55:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-9", "mode": "writeback"} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2824600108' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1333541928' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi107-77431-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: pgmap v86: 552 pgs: 32 creating+peering, 1 active+clean+snaptrim, 128 unknown, 391 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:57.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-9"}]': finished 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"}]': finished 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: osdmap e94: 8 total, 8 up, 8 in 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-9", "mode": "writeback"} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2824600108' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1333541928' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi107-77431-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:57.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: pgmap v86: 552 pgs: 32 creating+peering, 1 active+clean+snaptrim, 128 unknown, 391 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-21T11:55:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:55:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-9"}]': finished 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-77958-4", "tierpool": "test-rados-api-smithi107-77958-6"}]': finished 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: osdmap e94: 8 total, 8 up, 8 in 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-9", "mode": "writeback"} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2824600108' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/909029288' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1333541928' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattr_smithi107-77431-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:57.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:55:57.233 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-21T11:55:57.233 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] Global test environment set-up. 2024-09-21T11:55:57.233 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-21T11:55:57.233 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-21T11:55:57.233 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5053 ms) 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.RegisterLate (52 ms) 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: cluster: 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: id: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: health: HEALTH_WARN 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 12 pool(s) do not have an application enabled 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: services: 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mgr: y(active, since 87s), standbys: x 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-21T11:55:57.234 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: data: 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pools: 26 pools, 700 pgs 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: objects: 406 objects, 609 KiB 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: usage: 263 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pgs: 30.857% pgs not active 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 484 active+clean 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 216 creating+peering 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: io: 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: client: 8.7 KiB/s rd, 133 KiB/s wr, 171 op/s rd, 338 op/s wr 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: cluster: 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: id: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: health: HEALTH_WARN 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 15 pool(s) do not have an application enabled 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: services: 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mgr: y(active, since 89s), standbys: x 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-21T11:55:57.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: data: 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pools: 31 pools, 884 pgs 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: objects: 239 objects, 586 KiB 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: usage: 263 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pgs: 52.489% pgs unknown 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 7.240% pgs not active 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 464 unknown 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 356 active+clean 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 64 creating+peering 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: io: 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: client: 3.0 KiB/s rd, 21 KiB/s wr, 161 op/s rd, 192 op/s wr 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2186 ms) 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.Status (20022 ms) 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] 4 tests from LibRadosService (27313 ms total) 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] Global test environment tear-down 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27313 ms total) 2024-09-21T11:55:57.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ PASSED ] 4 tests. 2024-09-21T11:55:57.714 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:57.714 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-9", "mode": "writeback"}]': finished 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1333541928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-77431-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: osdmap e95: 8 total, 8 up, 8 in 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-77776-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-9", "mode": "writeback"}]': finished 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1333541928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-77431-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: osdmap e95: 8 total, 8 up, 8 in 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-77776-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:58.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-9", "mode": "writeback"}]': finished 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi107-77958-6", "pool2": "test-rados-api-smithi107-77958-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-77452-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1333541928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-77431-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: osdmap e95: 8 total, 8 up, 8 in 2024-09-21T11:55:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-77776-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:55:58.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:55:58.307 INFO:tasks.workunit.client.0.smithi107.stdout:ch_notify_pp: flushed 2024-09-21T11:55:58.307 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3003 ms) 2024-09-21T11:55:58.307 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-21T11:55:58.307 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: trying... 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185062672 notify_id 356482285574 notifier_gid 25180 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: timed out 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushed 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3006 ms) 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: List watches 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185046784 notify_id 369367187460 notifier_gid 25180 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 done 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: watch_check 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: unwatch2 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: done 2024-09-21T11:55:58.308 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3024 ms) 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: List watches 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94584185046784 notify_id 382252089351 notifier_gid 25180 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 done 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: watch_check 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: unwatch2 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: done 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3091 ms) 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (15639 ms total) 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (28442 ms total) 2024-09-21T11:55:58.309 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-21T11:55:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[30293]: pgmap v89: 612 pgs: 32 creating+peering, 1 active+clean+snaptrim, 224 unknown, 355 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 op/s 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[30293]: osdmap e96: 8 total, 8 up, 8 in 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9"} : dispatch 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[41097]: pgmap v89: 612 pgs: 32 creating+peering, 1 active+clean+snaptrim, 224 unknown, 355 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 op/s 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[41097]: osdmap e96: 8 total, 8 up, 8 in 2024-09-21T11:55:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9"} : dispatch 2024-09-21T11:55:59.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:55:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:55:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:58 smithi193 ceph-mon[33333]: pgmap v89: 612 pgs: 32 creating+peering, 1 active+clean+snaptrim, 224 unknown, 355 active+clean; 145 MiB data, 741 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 op/s 2024-09-21T11:55:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:55:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:55:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:58 smithi193 ceph-mon[33333]: osdmap e96: 8 total, 8 up, 8 in 2024-09-21T11:55:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9"} : dispatch 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-77776-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9"}]': finished 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: osdmap e97: 8 total, 8 up, 8 in 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3057682217' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2491777377' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-77776-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9"}]': finished 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: osdmap e97: 8 total, 8 up, 8 in 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3057682217' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2491777377' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:55:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-77776-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-9"}]': finished 2024-09-21T11:56:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: osdmap e97: 8 total, 8 up, 8 in 2024-09-21T11:56:00.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3057682217' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2491777377' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:55:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5061 ms) 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-21T11:56:00.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (76 ms) 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20061 ms) 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: attempt 0 of 20 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: failed to deregister: 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: { 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "epoch": 19, 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "modified": "2024-09-21T11:55:57.297663+0000", 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "services": { 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "laundry": { 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "daemons": { 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "summary": "", 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "close-test-pid78115": { 2024-09-21T11:56:00.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_epoch": 18, 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_stamp": "2024-09-21T11:55:55.178215+0000", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "gid": 31249, 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "addr": "172.21.15.107:0/3508120691", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "metadata": { 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "arch": "x86_64", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_release": "squid", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version_short": "19.3.0-5103-g5c1cdc3c", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro": "centos", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_version": "9", 2024-09-21T11:56:00.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "foo": "bar", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "hostname": "smithi107", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_total_kb": "32486960", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "os": "Linux", 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "this": "that" 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: }, 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "task_status": {} 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: }, 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "rgw": { 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "daemons": { 2024-09-21T11:56:00.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "summary": "", 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "24368": { 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_epoch": 12, 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_stamp": "2024-09-21T11:53:44.484291+0000", 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "gid": 24368, 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "addr": "172.21.15.107:0/4164906608", 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "metadata": { 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "arch": "x86_64", 2024-09-21T11:56:00.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_release": "squid", 2024-09-21T11:56:01.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: pgmap v92: 460 pgs: 72 unknown, 388 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1286369092' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: osdmap e98: 8 total, 8 up, 8 in 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: pgmap v92: 460 pgs: 72 unknown, 388 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1286369092' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: osdmap e98: 8 total, 8 up, 8 in 2024-09-21T11:56:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: pgmap v92: 460 pgs: 72 unknown, 388 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1286369092' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-77431-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-77452-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: osdmap e98: 8 total, 8 up, 8 in 2024-09-21T11:56:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:02.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:02.009 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:02.009 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:02.009 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[41097]: osdmap e99: 8 total, 8 up, 8 in 2024-09-21T11:56:02.009 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:02.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:02.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:02.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:02.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[30293]: osdmap e99: 8 total, 8 up, 8 in 2024-09-21T11:56:02.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:01 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:01 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77768-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:01 smithi193 ceph-mon[33333]: osdmap e99: 8 total, 8 up, 8 in 2024-09-21T11:56:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf135 misc: Running main() from gmock_main.cc 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [----------] Global test environment set-up. 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.Version (1711 ms) 2024-09-21T11:56:02.284 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (2065 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.LongName (2979 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3014 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2382 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3027 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.Exec (3012 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3029 ms) 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-09-21T11:56:02.285 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (2954 ms) 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2986 ms) 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2013 ms) 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2999 ms) 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [----------] 12 tests from NeoRadosMisc (32171 ms total) 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [----------] Global test environment tear-down 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [==========] 12 tests from 1 test suite ran. (32172 ms total) 2024-09-21T11:56:02.286 INFO:tasks.workunit.client.0.smithi107.stdout: misc: [ PASSED ] 12 tests. 2024-09-21T11:56:02.719 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:56:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: pgmap v95: 524 pgs: 200 unknown, 324 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: osdmap e100: 8 total, 8 up, 8 in 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3105151684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""} : dispatch 2024-09-21T11:56:03.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4231479667' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: pgmap v95: 524 pgs: 200 unknown, 324 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: osdmap e100: 8 total, 8 up, 8 in 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3105151684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""} : dispatch 2024-09-21T11:56:03.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4231479667' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""} : dispatch 2024-09-21T11:56:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: pgmap v95: 524 pgs: 200 unknown, 324 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-09-21T11:56:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: osdmap e100: 8 total, 8 up, 8 in 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3105151684' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""} : dispatch 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4231479667' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""} : dispatch 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""}]': finished 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: osdmap e101: 8 total, 8 up, 8 in 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""}]': finished 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: osdmap e101: 8 total, 8 up, 8 in 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-77452-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-77431-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache", "force_nonempty":""}]': finished 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: osdmap e101: 8 total, 8 up, 8 in 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:04.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:05.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: pgmap v98: 588 pgs: 264 unknown, 324 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: osdmap e102: 8 total, 8 up, 8 in 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-11"} : dispatch 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"} : dispatch 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"} : dispatch 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: pgmap v98: 588 pgs: 264 unknown, 324 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:05.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:56:05.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: osdmap e102: 8 total, 8 up, 8 in 2024-09-21T11:56:05.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-11"} : dispatch 2024-09-21T11:56:05.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"} : dispatch 2024-09-21T11:56:05.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"} : dispatch 2024-09-21T11:56:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: pgmap v98: 588 pgs: 264 unknown, 324 active+clean; 145 MiB data, 795 MiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77768-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:56:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: osdmap e102: 8 total, 8 up, 8 in 2024-09-21T11:56:05.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-11"} : dispatch 2024-09-21T11:56:05.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2486221462' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"} : dispatch 2024-09-21T11:56:05.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"} : dispatch 2024-09-21T11:56:05.392 INFO:tasks.workunit.client.0.smithi107.stdout:09a046d301) squid (dev)", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version_short": "19.3.0-5103-g5c1cdc3c", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "container_hostname": "smithi107", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro": "centos", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_version": "9", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "hostname": "smithi107", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "id": "foo.a", 2024-09-21T11:56:05.393 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_total_kb": "32486960", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "num_handles": "1", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "os": "Linux", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "pid": "7", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "realm_id": "", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "realm_name": "", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zone_id": "4370ac67-3ceb-4268-89eb-9b844c5bfc0a", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zone_name": "default", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zonegroup_id": "59ed68f1-5551-4f3d-acde-82e771663b47", 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zonegroup_name": "default" 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: }, 2024-09-21T11:56:05.394 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "task_status": {} 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: attempt 1 of 20 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: failed to deregister: 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: { 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "epoch": 21, 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "modified": "2024-09-21T11:56:01.298908+0000", 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "services": { 2024-09-21T11:56:05.395 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "laundry": { 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "daemons": { 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "summary": "", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "close-test-pid78115": { 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_epoch": 21, 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_stamp": "2024-09-21T11:56:00.360495+0000", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "gid": 35698, 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "addr": "172.21.15.107:0/2510859557", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "metadata": { 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "arch": "x86_64", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_release": "squid", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version_short": "19.3.0-5103-g5c1cdc3c", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro": "centos", 2024-09-21T11:56:05.396 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_version": "9", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "foo": "bar", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "hostname": "smithi107", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_total_kb": "32486960", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "os": "Linux", 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "this": "that" 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: }, 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "task_status": {} 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.397 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: }, 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "rgw": { 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "daemons": { 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "summary": "", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "24368": { 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_epoch": 12, 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "start_stamp": "2024-09-21T11:53:44.484291+0000", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "gid": 24368, 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "addr": "172.21.15.107:0/4164906608", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "metadata": { 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "arch": "x86_64", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_release": "squid", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "ceph_version_short": "19.3.0-5103-g5c1cdc3c", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "container_hostname": "smithi107", 2024-09-21T11:56:05.398 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro": "centos", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "distro_version": "9", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "hostname": "smithi107", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "id": "foo.a", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "mem_total_kb": "32486960", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "num_handles": "1", 2024-09-21T11:56:05.399 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "os": "Linux", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "pid": "7", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "realm_id": "", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "realm_name": "", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zone_id": "4370ac67-3ceb-4268-89eb-9b844c5bfc0a", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zone_name": "default", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zonegroup_id": "59ed68f1-5551-4f3d-acde-82e771663b47", 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "zonegroup_name": "default" 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: }, 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: "task_status": {} 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:05.400 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-21T11:56:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-21T11:56:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-21T11:56:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-21T11:56:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-21T11:56:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-21T11:56:06.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: 17.0 deep-scrub starts 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: 17.2 deep-scrub starts 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: 17.0 deep-scrub ok 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-11"}]': finished 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"}]': finished 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: osdmap e103: 8 total, 8 up, 8 in 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-11", "mode": "writeback"} : dispatch 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3277981945' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2092383430' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi107-77431-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2546334611' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: 17.9 deep-scrub starts 2024-09-21T11:56:06.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[41097]: 17.9 deep-scrub ok 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-21T11:56:06.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: 17.0 deep-scrub starts 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: 17.2 deep-scrub starts 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: 17.0 deep-scrub ok 2024-09-21T11:56:06.008 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-11"}]': finished 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"}]': finished 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: osdmap e103: 8 total, 8 up, 8 in 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-11", "mode": "writeback"} : dispatch 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3277981945' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2092383430' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi107-77431-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2546334611' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: 17.9 deep-scrub starts 2024-09-21T11:56:06.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:05 smithi107 ceph-mon[30293]: 17.9 deep-scrub ok 2024-09-21T11:56:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.0"} : dispatch 2024-09-21T11:56:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.1"} : dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.2"} : dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.3"} : dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.4"} : dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.5"} : dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.6"} : dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-21T11:56:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.7"} : dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.8"} : dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "17.9"} : dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: 17.0 deep-scrub starts 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: 17.2 deep-scrub starts 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: 17.0 deep-scrub ok 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-11"}]': finished 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77768-10", "tierpool":"test-rados-api-smithi107-77768-10-cache"}]': finished 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: osdmap e103: 8 total, 8 up, 8 in 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-11", "mode": "writeback"} : dispatch 2024-09-21T11:56:06.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3277981945' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-9","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:06.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2092383430' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafe_smithi107-77431-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:06.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2546334611' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:06.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: 17.9 deep-scrub starts 2024-09-21T11:56:06.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:05 smithi193 ceph-mon[33333]: 17.9 deep-scrub ok 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.2 deep-scrub ok 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: pgmap v101: 524 pgs: 64 unknown, 12 creating+activating, 448 active+clean; 145 MiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.1 deep-scrub starts 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.1 deep-scrub ok 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.5 deep-scrub starts 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.5 deep-scrub ok 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.8 deep-scrub starts 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: 17.8 deep-scrub ok 2024-09-21T11:56:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-11", "mode": "writeback"}]': finished 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3277981945' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2092383430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-77431-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[30293]: osdmap e104: 8 total, 8 up, 8 in 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.2 deep-scrub ok 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: pgmap v101: 524 pgs: 64 unknown, 12 creating+activating, 448 active+clean; 145 MiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.1 deep-scrub starts 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.1 deep-scrub ok 2024-09-21T11:56:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.5 deep-scrub starts 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.5 deep-scrub ok 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.8 deep-scrub starts 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: 17.8 deep-scrub ok 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-11", "mode": "writeback"}]': finished 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3277981945' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2092383430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-77431-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:07.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:06 smithi107 ceph-mon[41097]: osdmap e104: 8 total, 8 up, 8 in 2024-09-21T11:56:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.2 deep-scrub ok 2024-09-21T11:56:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: pgmap v101: 524 pgs: 64 unknown, 12 creating+activating, 448 active+clean; 145 MiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-09-21T11:56:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.1 deep-scrub starts 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.1 deep-scrub ok 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.5 deep-scrub starts 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.5 deep-scrub ok 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.8 deep-scrub starts 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: 17.8 deep-scrub ok 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-11", "mode": "writeback"}]': finished 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3277981945' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2092383430' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-77431-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:06 smithi193 ceph-mon[33333]: osdmap e104: 8 total, 8 up, 8 in 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: Running main() from gmock_main.cc 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] Global test environment set-up. 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolList (3061 ms) 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-21T11:56:07.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3073 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (2988 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3036 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (2370 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5035 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (4983 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5002 ms) 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-21T11:56:07.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7987 ms) 2024-09-21T11:56:07.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] 9 tests from LibRadosPools (37535 ms total) 2024-09-21T11:56:07.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: 2024-09-21T11:56:07.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] Global test environment tear-down 2024-09-21T11:56:07.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (37537 ms total) 2024-09-21T11:56:07.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: 17.3 deep-scrub starts 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: 17.3 deep-scrub ok 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: 17.4 deep-scrub starts 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: 17.4 deep-scrub ok 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: 17.6 deep-scrub starts 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: 17.6 deep-scrub ok 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: osdmap e105: 8 total, 8 up, 8 in 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: 17.3 deep-scrub starts 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: 17.3 deep-scrub ok 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: 17.4 deep-scrub starts 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: 17.4 deep-scrub ok 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: 17.6 deep-scrub starts 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: 17.6 deep-scrub ok 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: osdmap e105: 8 total, 8 up, 8 in 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:56:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:07 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: 17.3 deep-scrub starts 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: 17.3 deep-scrub ok 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: 17.4 deep-scrub starts 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: 17.4 deep-scrub ok 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: 17.6 deep-scrub starts 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: 17.6 deep-scrub ok 2024-09-21T11:56:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: osdmap e105: 8 total, 8 up, 8 in 2024-09-21T11:56:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:56:08.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:07 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:08.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] Global test environment set-up. 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-21T11:55:29.776+0000 7f25ed4f1780 -1 failed for service _ceph-mon._tcp 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-21T11:55:29.777+0000 7f25ed4f1780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (67 ms) 2024-09-21T11:56:08.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5011 ms) 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5079 ms total) 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: started 0x55d0176098e0 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: started 0x55d017746360 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: started 2 aios 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: waiting 0x55d0176098e0 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: waiting 0x55d017746360 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: done. 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (6071 ms) 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (6071 ms total) 2024-09-21T11:56:08.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.Exec (87 ms) 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4939 ms) 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-21T11:56:08.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (20813 ms) 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 8 tests from LibRadosMisc (25845 ms total) 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] Global test environment tear-down 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (38547 ms total) 2024-09-21T11:56:08.291 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-21T11:56:09.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:56:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: pgmap v104: 460 pgs: 96 unknown, 364 active+clean; 145 MiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-09-21T11:56:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: 17.7 deep-scrub starts 2024-09-21T11:56:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: 17.7 deep-scrub ok 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: osdmap e106: 8 total, 8 up, 8 in 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11"} : dispatch 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1296954103' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-77431-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2557616982' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:08 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: pgmap v104: 460 pgs: 96 unknown, 364 active+clean; 145 MiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: 17.7 deep-scrub starts 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: 17.7 deep-scrub ok 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: osdmap e106: 8 total, 8 up, 8 in 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11"} : dispatch 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1296954103' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-77431-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2557616982' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: pgmap v104: 460 pgs: 96 unknown, 364 active+clean; 145 MiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 3 op/s 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: 17.7 deep-scrub starts 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: 17.7 deep-scrub ok 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: osdmap e106: 8 total, 8 up, 8 in 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11"} : dispatch 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1296954103' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-77431-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2557616982' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:08 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11"}]': finished 2024-09-21T11:56:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1296954103' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-77431-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: osdmap e107: 8 total, 8 up, 8 in 2024-09-21T11:56:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11"}]': finished 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1296954103' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-77431-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: osdmap e107: 8 total, 8 up, 8 in 2024-09-21T11:56:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-11"}]': finished 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1296954103' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-77431-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-77452-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: osdmap e107: 8 total, 8 up, 8 in 2024-09-21T11:56:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: } 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: attempt 2 of 20 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (15375 ms) 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (40573 ms total) 2024-09-21T11:56:10.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: 2024-09-21T11:56:10.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-21T11:56:10.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (40573 ms total) 2024-09-21T11:56:10.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-21T11:56:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: pgmap v107: 484 pgs: 64 unknown, 64 creating+peering, 7 creating+activating, 349 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-21T11:56:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: osdmap e108: 8 total, 8 up, 8 in 2024-09-21T11:56:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:56:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3186199776' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: pgmap v107: 484 pgs: 64 unknown, 64 creating+peering, 7 creating+activating, 349 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: osdmap e108: 8 total, 8 up, 8 in 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3186199776' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: pgmap v107: 484 pgs: 64 unknown, 64 creating+peering, 7 creating+activating, 349 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-09-21T11:56:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: osdmap e108: 8 total, 8 up, 8 in 2024-09-21T11:56:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"} : dispatch 2024-09-21T11:56:11.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3186199776' entity='client.admin' cmd={"prefix": "service dump"} : dispatch 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: Running main() from gmock_main.cc 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [----------] Global test environment set-up. 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [----------] 14 tests from NeoRadosIo 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.Limits (2811 ms) 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3058 ms) 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-09-21T11:56:11.282 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.ReadOp (2999 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.SparseRead (3016 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.RoundTrip (2378 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3011 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3045 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (2946 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3031 ms) 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-09-21T11:56:11.283 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.Trunc (2990 ms) 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.Remove (3021 ms) 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3006 ms) 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.RmXattr (2952 ms) 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3013 ms) 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [----------] 14 tests from NeoRadosIo (41278 ms total) 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [----------] Global test environment tear-down 2024-09-21T11:56:11.284 INFO:tasks.workunit.client.0.smithi107.stdout: io: [==========] 14 tests from 1 test suite ran. (41278 ms total) 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: io: [ PASSED ] 14 tests. 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: Running main() from gmock_main.cc 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [----------] Global test environment set-up. 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2683 ms) 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-09-21T11:56:11.285 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3080 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (2988 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3010 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (2377 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3015 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3045 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2953 ms) 2024-09-21T11:56:11.286 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3039 ms) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2980 ms) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2969 ms) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3023 ms) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2981 ms) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3013 ms) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (41156 ms total) 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: 2024-09-21T11:56:11.287 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [----------] Global test environment tear-down 2024-09-21T11:56:11.288 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (41156 ms total) 2024-09-21T11:56:11.288 INFO:tasks.workunit.client.0.smithi107.stdout: read_operations: [ PASSED ] 14 tests. 2024-09-21T11:56:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: osdmap e109: 8 total, 8 up, 8 in 2024-09-21T11:56:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1685585245' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-77452-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3627544911' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: osdmap e109: 8 total, 8 up, 8 in 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1685585245' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-77452-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3627544911' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:12.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/326610310' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-77776-10"}]': finished 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: osdmap e109: 8 total, 8 up, 8 in 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1685585245' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-77452-11","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3627544911' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:12.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:12.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:12.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:56:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: pgmap v110: 420 pgs: 96 unknown, 7 creating+activating, 317 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-09-21T11:56:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1685585245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-77452-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: osdmap e110: 8 total, 8 up, 8 in 2024-09-21T11:56:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:13.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: pgmap v110: 420 pgs: 96 unknown, 7 creating+activating, 317 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1685585245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-77452-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: osdmap e110: 8 total, 8 up, 8 in 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: pgmap v110: 420 pgs: 96 unknown, 7 creating+activating, 317 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1685585245' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-77452-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi107-77431-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: osdmap e110: 8 total, 8 up, 8 in 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:13.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:13 smithi193 ceph-mon[33333]: osdmap e111: 8 total, 8 up, 8 in 2024-09-21T11:56:14.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:13 smithi107 ceph-mon[30293]: osdmap e111: 8 total, 8 up, 8 in 2024-09-21T11:56:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:13 smithi107 ceph-mon[41097]: osdmap e111: 8 total, 8 up, 8 in 2024-09-21T11:56:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: pgmap v113: 356 pgs: 32 unknown, 7 creating+activating, 317 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: osdmap e112: 8 total, 8 up, 8 in 2024-09-21T11:56:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/440938899' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/631180772' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-77452-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:14 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:56:15.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: pgmap v113: 356 pgs: 32 unknown, 7 creating+activating, 317 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: osdmap e112: 8 total, 8 up, 8 in 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/440938899' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/631180772' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-77452-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: pgmap v113: 356 pgs: 32 unknown, 7 creating+activating, 317 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-77776-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: osdmap e112: 8 total, 8 up, 8 in 2024-09-21T11:56:15.115 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/440938899' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/631180772' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-77452-12","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:15.116 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:14 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:56:16.049 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:16.049 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:16.049 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/631180772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-77452-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:16.050 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[41097]: osdmap e113: 8 total, 8 up, 8 in 2024-09-21T11:56:16.050 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:16.050 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:16.050 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/631180772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-77452-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:16.050 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:15 smithi107 ceph-mon[30293]: osdmap e113: 8 total, 8 up, 8 in 2024-09-21T11:56:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:15 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-77431-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/631180772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-77452-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:15 smithi193 ceph-mon[33333]: osdmap e113: 8 total, 8 up, 8 in 2024-09-21T11:56:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:16 smithi193 ceph-mon[33333]: pgmap v116: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 39 KiB/s wr, 39 op/s 2024-09-21T11:56:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:16 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:16 smithi193 ceph-mon[33333]: osdmap e114: 8 total, 8 up, 8 in 2024-09-21T11:56:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:17.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[41097]: pgmap v116: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 39 KiB/s wr, 39 op/s 2024-09-21T11:56:17.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[41097]: osdmap e114: 8 total, 8 up, 8 in 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[30293]: pgmap v116: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 39 KiB/s wr, 39 op/s 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[30293]: osdmap e114: 8 total, 8 up, 8 in 2024-09-21T11:56:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:56:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: osdmap e115: 8 total, 8 up, 8 in 2024-09-21T11:56:18.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-13"} : dispatch 2024-09-21T11:56:18.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3152793443' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-77431-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3715880313' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: osdmap e115: 8 total, 8 up, 8 in 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-13"} : dispatch 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3152793443' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-77431-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3715880313' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:18.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:56:18.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: osdmap e115: 8 total, 8 up, 8 in 2024-09-21T11:56:18.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-13"} : dispatch 2024-09-21T11:56:18.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3152793443' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-77431-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3715880313' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:18.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:19.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: pgmap v119: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 39 KiB/s wr, 39 op/s 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-13"}]': finished 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3152793443' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-77431-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: osdmap e116: 8 total, 8 up, 8 in 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-13", "mode": "writeback"} : dispatch 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:19.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:18 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:19.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: pgmap v119: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 39 KiB/s wr, 39 op/s 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-13"}]': finished 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3152793443' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-77431-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: osdmap e116: 8 total, 8 up, 8 in 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-13", "mode": "writeback"} : dispatch 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:19.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: pgmap v119: 428 pgs: 104 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 39 KiB/s wr, 39 op/s 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-13"}]': finished 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3152793443' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-77431-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-77452-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: osdmap e116: 8 total, 8 up, 8 in 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-13", "mode": "writeback"} : dispatch 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:19.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:18 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:56:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-13", "mode": "writeback"}]': finished 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: osdmap e117: 8 total, 8 up, 8 in 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:19 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-13", "mode": "writeback"}]': finished 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: osdmap e117: 8 total, 8 up, 8 in 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-13", "mode": "writeback"}]': finished 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: osdmap e117: 8 total, 8 up, 8 in 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:20.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:19 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: pgmap v122: 364 pgs: 364 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 345 KiB/s wr, 87 op/s 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.0"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.1"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.2"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.3"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.4"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.5"} : dispatch 2024-09-21T11:56:21.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.6"} : dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.7"} : dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.8"} : dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.9"} : dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.1 scrub starts 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.1 scrub ok 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.8 scrub starts 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.8 scrub ok 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-21T11:56:21.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: osdmap e118: 8 total, 8 up, 8 in 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2125779755' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-77431-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2513388394' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.4 scrub starts 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.4 scrub ok 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.2 scrub starts 2024-09-21T11:56:21.053 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:20 smithi193 ceph-mon[33333]: 172.2 scrub ok 2024-09-21T11:56:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: pgmap v122: 364 pgs: 364 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 345 KiB/s wr, 87 op/s 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.0"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.1"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.2"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.3"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.4"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.5"} : dispatch 2024-09-21T11:56:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.6"} : dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.7"} : dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.8"} : dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.9"} : dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.1 scrub starts 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.1 scrub ok 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.8 scrub starts 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.8 scrub ok 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: osdmap e118: 8 total, 8 up, 8 in 2024-09-21T11:56:21.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2125779755' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-77431-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: pgmap v122: 364 pgs: 364 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 345 KiB/s wr, 87 op/s 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.0"} : dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.1"} : dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.2"} : dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.3"} : dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-09-21T11:56:21.257 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.4"} : dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.5"} : dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.6"} : dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.7"} : dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.8"} : dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "172.9"} : dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.1 scrub starts 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.1 scrub ok 2024-09-21T11:56:21.258 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.8 scrub starts 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.8 scrub ok 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: osdmap e118: 8 total, 8 up, 8 in 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2125779755' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-77431-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2513388394' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.4 scrub starts 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.4 scrub ok 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.2 scrub starts 2024-09-21T11:56:21.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[30293]: 172.2 scrub ok 2024-09-21T11:56:21.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2513388394' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:21.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.4 scrub starts 2024-09-21T11:56:21.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.4 scrub ok 2024-09-21T11:56:21.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.2 scrub starts 2024-09-21T11:56:21.260 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:20 smithi107 ceph-mon[41097]: 172.2 scrub ok 2024-09-21T11:56:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.7 scrub starts 2024-09-21T11:56:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.7 scrub ok 2024-09-21T11:56:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.0 scrub starts 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.0 scrub ok 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.9 scrub starts 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.9 scrub ok 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.5 scrub starts 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.5 scrub ok 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2125779755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-77431-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: osdmap e119: 8 total, 8 up, 8 in 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.3 scrub starts 2024-09-21T11:56:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:21 smithi193 ceph-mon[33333]: 172.3 scrub ok 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.7 scrub starts 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.7 scrub ok 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.0 scrub starts 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.0 scrub ok 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.9 scrub starts 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.9 scrub ok 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.5 scrub starts 2024-09-21T11:56:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.5 scrub ok 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2125779755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-77431-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: osdmap e119: 8 total, 8 up, 8 in 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.3 scrub starts 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[41097]: 172.3 scrub ok 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.7 scrub starts 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.7 scrub ok 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.0 scrub starts 2024-09-21T11:56:22.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.0 scrub ok 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.9 scrub starts 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.9 scrub ok 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.5 scrub starts 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.5 scrub ok 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2125779755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-77431-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-77452-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: osdmap e119: 8 total, 8 up, 8 in 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"} : dispatch 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.3 scrub starts 2024-09-21T11:56:22.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:21 smithi107 ceph-mon[30293]: 172.3 scrub ok 2024-09-21T11:56:22.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:56:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: pgmap v125: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 89 KiB/s wr, 86 op/s 2024-09-21T11:56:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: 172.6 scrub starts 2024-09-21T11:56:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: 172.6 scrub ok 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"}]': finished 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: osdmap e120: 8 total, 8 up, 8 in 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:56:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:22 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:23.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: pgmap v125: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 89 KiB/s wr, 86 op/s 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: 172.6 scrub starts 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: 172.6 scrub ok 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"}]': finished 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: osdmap e120: 8 total, 8 up, 8 in 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:56:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: pgmap v125: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 89 KiB/s wr, 86 op/s 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: 172.6 scrub starts 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: 172.6 scrub ok 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pg_num","val":"11"}]': finished 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: osdmap e120: 8 total, 8 up, 8 in 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"} : dispatch 2024-09-21T11:56:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:22 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: osdmap e121: 8 total, 8 up, 8 in 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4228014872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2744345361' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:24 smithi193 ceph-mon[33333]: pgmap v128: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: osdmap e121: 8 total, 8 up, 8 in 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4228014872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2744345361' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[30293]: pgmap v128: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-77529-1","var":"pgp_num","val":"11"}]': finished 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: osdmap e121: 8 total, 8 up, 8 in 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4228014872' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2744345361' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:24.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:24.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:24 smithi107 ceph-mon[41097]: pgmap v128: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2056 ms) 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-21T11:56:25.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2011 ms) 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2001 ms) 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1999 ms) 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8068 ms total) 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4124 ms) 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4071 ms) 2024-09-21T11:56:25.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4908 ms) 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13103 ms total) 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2991 ms) 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2020 ms) 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2027 ms) 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1958 ms) 2024-09-21T11:56:25.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8996 ms total) 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4030 ms) 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4006 ms) 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8036 ms total) 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (55539 ms total) 2024-09-21T11:56:25.290 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-09-21T11:56:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:25.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:25.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:25.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:25.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:25.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: osdmap e122: 8 total, 8 up, 8 in 2024-09-21T11:56:25.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: osdmap e122: 8 total, 8 up, 8 in 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-77452-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-77431-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2761764860' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: osdmap e122: 8 total, 8 up, 8 in 2024-09-21T11:56:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"} : dispatch 2024-09-21T11:56:26.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: osdmap e123: 8 total, 8 up, 8 in 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: pgmap v131: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 73 op/s; 31 B/s, 0 objects/s recovering 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: osdmap e124: 8 total, 8 up, 8 in 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3891994228' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-77452-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4216458682' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2204010254' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: osdmap e123: 8 total, 8 up, 8 in 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: pgmap v131: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 73 op/s; 31 B/s, 0 objects/s recovering 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-21T11:56:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: osdmap e124: 8 total, 8 up, 8 in 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3891994228' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-77452-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4216458682' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2204010254' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-77776-15"}]': finished 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: osdmap e123: 8 total, 8 up, 8 in 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: pgmap v131: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 73 op/s; 31 B/s, 0 objects/s recovering 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: osdmap e124: 8 total, 8 up, 8 in 2024-09-21T11:56:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3891994228' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-77452-16","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4216458682' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:26.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2204010254' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3891994228' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-77452-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:27 smithi193 ceph-mon[33333]: osdmap e125: 8 total, 8 up, 8 in 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3891994228' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-77452-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[30293]: osdmap e125: 8 total, 8 up, 8 in 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3891994228' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-77452-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77431-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77822-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:27 smithi107 ceph-mon[41097]: osdmap e125: 8 total, 8 up, 8 in 2024-09-21T11:56:28.579 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:28 smithi193 ceph-mon[33333]: pgmap v134: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 74 op/s; 31 B/s, 0 objects/s recovering 2024-09-21T11:56:28.580 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:28.580 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:28 smithi193 ceph-mon[33333]: osdmap e126: 8 total, 8 up, 8 in 2024-09-21T11:56:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:28 smithi107 ceph-mon[30293]: pgmap v134: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 74 op/s; 31 B/s, 0 objects/s recovering 2024-09-21T11:56:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:28 smithi107 ceph-mon[30293]: osdmap e126: 8 total, 8 up, 8 in 2024-09-21T11:56:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:28 smithi107 ceph-mon[41097]: pgmap v134: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 74 op/s; 31 B/s, 0 objects/s recovering 2024-09-21T11:56:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:28 smithi107 ceph-mon[41097]: osdmap e126: 8 total, 8 up, 8 in 2024-09-21T11:56:29.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:56:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: osdmap e127: 8 total, 8 up, 8 in 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2836601502' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2035713789' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: osdmap e127: 8 total, 8 up, 8 in 2024-09-21T11:56:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2836601502' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2035713789' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: osdmap e127: 8 total, 8 up, 8 in 2024-09-21T11:56:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2836601502' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2035713789' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:29.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[30293]: pgmap v137: 419 pgs: 64 unknown, 1 peering, 354 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[30293]: osdmap e128: 8 total, 8 up, 8 in 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[41097]: pgmap v137: 419 pgs: 64 unknown, 1 peering, 354 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:30 smithi107 ceph-mon[41097]: osdmap e128: 8 total, 8 up, 8 in 2024-09-21T11:56:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:30 smithi193 ceph-mon[33333]: pgmap v137: 419 pgs: 64 unknown, 1 peering, 354 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:56:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi107-77431-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:30 smithi193 ceph-mon[33333]: osdmap e128: 8 total, 8 up, 8 in 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: osdmap e129: 8 total, 8 up, 8 in 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: osdmap e129: 8 total, 8 up, 8 in 2024-09-21T11:56:31.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: osdmap e129: 8 total, 8 up, 8 in 2024-09-21T11:56:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:32.702 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:32 smithi193 ceph-mon[33333]: pgmap v140: 323 pgs: 1 peering, 322 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:56:32.772 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:32.773 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:32 smithi193 ceph-mon[33333]: osdmap e130: 8 total, 8 up, 8 in 2024-09-21T11:56:32.773 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1989265576' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.773 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.773 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/651974963' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-77452-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.773 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:56:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[30293]: pgmap v140: 323 pgs: 1 peering, 322 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[30293]: osdmap e130: 8 total, 8 up, 8 in 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1989265576' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/651974963' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-77452-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[41097]: pgmap v140: 323 pgs: 1 peering, 322 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[41097]: osdmap e130: 8 total, 8 up, 8 in 2024-09-21T11:56:32.774 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1989265576' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.775 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:32.775 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/651974963' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-77452-18","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:33.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:33.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:33.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:33.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/651974963' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-77452-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:33.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:33 smithi193 ceph-mon[33333]: osdmap e131: 8 total, 8 up, 8 in 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/651974963' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-77452-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[41097]: osdmap e131: 8 total, 8 up, 8 in 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-77822-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-77431-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/651974963' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-77452-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:33 smithi107 ceph-mon[30293]: osdmap e131: 8 total, 8 up, 8 in 2024-09-21T11:56:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:34 smithi193 ceph-mon[33333]: pgmap v143: 395 pgs: 1 creating+peering, 3 creating+activating, 68 unknown, 1 peering, 322 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:34.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:34 smithi193 ceph-mon[33333]: osdmap e132: 8 total, 8 up, 8 in 2024-09-21T11:56:34.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:34 smithi107 ceph-mon[30293]: pgmap v143: 395 pgs: 1 creating+peering, 3 creating+activating, 68 unknown, 1 peering, 322 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:34.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:34 smithi107 ceph-mon[30293]: osdmap e132: 8 total, 8 up, 8 in 2024-09-21T11:56:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:34 smithi107 ceph-mon[41097]: pgmap v143: 395 pgs: 1 creating+peering, 3 creating+activating, 68 unknown, 1 peering, 322 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:34 smithi107 ceph-mon[41097]: osdmap e132: 8 total, 8 up, 8 in 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: osdmap e133: 8 total, 8 up, 8 in 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2"} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1875999960' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1123192753' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: osdmap e133: 8 total, 8 up, 8 in 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2"} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1875999960' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1123192753' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: osdmap e133: 8 total, 8 up, 8 in 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2"} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1875999960' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1123192753' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:35.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: pgmap v146: 395 pgs: 64 unknown, 8 creating+peering, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: osdmap e134: 8 total, 8 up, 8 in 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: pgmap v146: 395 pgs: 64 unknown, 8 creating+peering, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:36.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: osdmap e134: 8 total, 8 up, 8 in 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: pgmap v146: 395 pgs: 64 unknown, 8 creating+peering, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-77452-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-77431-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-77697-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: osdmap e134: 8 total, 8 up, 8 in 2024-09-21T11:56:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"dne","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"} : dispatch 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: seed 77822 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2056 ms) 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2012 ms) 2024-09-21T11:56:38.275 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2005 ms) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (1994 ms) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3390 ms) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11457 ms total) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4168 ms) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3833 ms) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5996 ms) 2024-09-21T11:56:38.276 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4002 ms) 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2092 ms) 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5103-g5c1cdc3c/rpm/el9/BUILD/ceph-19.3.0-5103-g5c1cdc3c/src/test/librados/snapshots_cxx.cc:460: Skipped 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi107-77822-7 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17931 ms) 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38022 ms total) 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-21T11:56:38.277 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (10 ms) 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5039 ms) 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5049 ms total) 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3011 ms) 2024-09-21T11:56:38.278 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-21T11:56:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-21T11:56:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: osdmap e135: 8 total, 8 up, 8 in 2024-09-21T11:56:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:56:38.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:56:38.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:38.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: pgmap v149: 331 pgs: 8 creating+peering, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:38.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:38.555 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-21T11:56:38.555 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: osdmap e135: 8 total, 8 up, 8 in 2024-09-21T11:56:38.555 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: pgmap v149: 331 pgs: 8 creating+peering, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: osdmap e135: 8 total, 8 up, 8 in 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: pgmap v149: 331 pgs: 8 creating+peering, 323 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:38.556 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:39.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:56:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:56:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-21T11:56:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: osdmap e136: 8 total, 8 up, 8 in 2024-09-21T11:56:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/292599978' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:56:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4032838386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-77431-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:56:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: osdmap e136: 8 total, 8 up, 8 in 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/292599978' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4032838386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-77431-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:56:39.575 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: osdmap e136: 8 total, 8 up, 8 in 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/292599978' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3901566850' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4032838386' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-77431-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"} : dispatch 2024-09-21T11:56:39.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:40.298 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRados api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-21T11:56:40.298 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-21T11:56:40.298 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-21T11:56:40.298 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: seed 77697 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (3 ms) 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (347 ms) 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (11 ms) 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-21T11:56:40.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (10 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (8 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (1 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (8 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-21T11:56:40.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (3 ms) 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (12 ms) 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1515 ms) 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (478 ms) 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: done waiting, doing copies 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: done waiting 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60644 ms) 2024-09-21T11:56:40.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (5 ms) 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4423 ms) 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-21T11:56:40.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-21T11:56:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4032838386' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-77431-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"}]': finished 2024-09-21T11:56:40.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:40 smithi193 ceph-mon[33333]: osdmap e137: 8 total, 8 up, 8 in 2024-09-21T11:56:40.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:40.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:40 smithi193 ceph-mon[33333]: pgmap v152: 395 pgs: 64 creating+peering, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:56:40.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4032838386' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-77431-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"}]': finished 2024-09-21T11:56:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[30293]: osdmap e137: 8 total, 8 up, 8 in 2024-09-21T11:56:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[30293]: pgmap v152: 395 pgs: 64 creating+peering, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:56:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-77452-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:40.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4032838386' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-77431-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:40.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-77697-1","app":"app1","key":"key1"}]': finished 2024-09-21T11:56:40.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[41097]: osdmap e137: 8 total, 8 up, 8 in 2024-09-21T11:56:40.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:40.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:40 smithi107 ceph-mon[41097]: pgmap v152: 395 pgs: 64 creating+peering, 331 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:56:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:41 smithi193 ceph-mon[33333]: osdmap e138: 8 total, 8 up, 8 in 2024-09-21T11:56:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:41.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77697-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:41.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[30293]: osdmap e138: 8 total, 8 up, 8 in 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77697-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[41097]: osdmap e138: 8 total, 8 up, 8 in 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77697-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:42.050 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 11:56:41 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=infra.usagestats t=2024-09-21T11:56:41.627462273Z level=info msg="Usage stats are ready to report" 2024-09-21T11:56:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:56:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77697-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: osdmap e139: 8 total, 8 up, 8 in 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1427172039' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77697-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3479066296' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-77431-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: pgmap v155: 363 pgs: 64 unknown, 299 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3479066296' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-77431-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[30293]: osdmap e140: 8 total, 8 up, 8 in 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77697-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: osdmap e139: 8 total, 8 up, 8 in 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1427172039' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77697-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3479066296' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-77431-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: pgmap v155: 363 pgs: 64 unknown, 299 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:42.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:42.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3479066296' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-77431-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:42.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:42 smithi107 ceph-mon[41097]: osdmap e140: 8 total, 8 up, 8 in 2024-09-21T11:56:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77697-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: osdmap e139: 8 total, 8 up, 8 in 2024-09-21T11:56:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1427172039' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77697-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3479066296' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-77431-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: pgmap v155: 363 pgs: 64 unknown, 299 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-77452-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3479066296' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-77431-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:42 smithi193 ceph-mon[33333]: osdmap e140: 8 total, 8 up, 8 in 2024-09-21T11:56:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77697-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[30293]: osdmap e141: 8 total, 8 up, 8 in 2024-09-21T11:56:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:43.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:43.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77697-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:43.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[41097]: osdmap e141: 8 total, 8 up, 8 in 2024-09-21T11:56:43.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:43.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:43.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77697-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:43 smithi193 ceph-mon[33333]: osdmap e141: 8 total, 8 up, 8 in 2024-09-21T11:56:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:43.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:43.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:44.280 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP snapshots: Running main() from gmock_main.cc 2024-09-21T11:56:44.280 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-09-21T11:56:44.280 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [----------] Global test environment set-up. 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4748 ms) 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (4998 ms) 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3389 ms) 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5058 ms) 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6964 ms) 2024-09-21T11:56:44.281 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (25157 ms total) 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5012 ms) 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6020 ms) 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7962 ms) 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6008 ms) 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4002 ms) 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-09-21T11:56:44.282 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi107-78817-11. 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: Waiting for snaps to purge. 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19997 ms) 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49001 ms total) 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [----------] Global test environment tear-down 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (74158 ms total) 2024-09-21T11:56:44.283 INFO:tasks.workunit.client.0.smithi107.stdout: snapshots: [ PASSED ] 11 tests. 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: pgmap v158: 299 pgs: 1 creating+peering, 7 unknown, 291 active+clean; 592 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: osdmap e142: 8 total, 8 up, 8 in 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3673495348' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77452-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2233142501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-77431-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: pgmap v158: 299 pgs: 1 creating+peering, 7 unknown, 291 active+clean; 592 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: osdmap e142: 8 total, 8 up, 8 in 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3673495348' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77452-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2233142501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-77431-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: pgmap v158: 299 pgs: 1 creating+peering, 7 unknown, 291 active+clean; 592 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1756362020' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: osdmap e142: 8 total, 8 up, 8 in 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"} : dispatch 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3673495348' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77452-22","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2233142501' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-77431-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:44.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3673495348' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77452-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2233142501' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-77431-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: osdmap e143: 8 total, 8 up, 8 in 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: pgmap v161: 363 pgs: 96 unknown, 8 creating+peering, 259 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:46.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:46 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3673495348' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77452-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2233142501' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-77431-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: osdmap e143: 8 total, 8 up, 8 in 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: pgmap v161: 363 pgs: 96 unknown, 8 creating+peering, 259 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:46.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-77822-16"}]': finished 2024-09-21T11:56:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3673495348' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-77452-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2233142501' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-77431-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: osdmap e143: 8 total, 8 up, 8 in 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: pgmap v161: 363 pgs: 96 unknown, 8 creating+peering, 259 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:46.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:46 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: osdmap e144: 8 total, 8 up, 8 in 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: osdmap e144: 8 total, 8 up, 8 in 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:47 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77697-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: osdmap e144: 8 total, 8 up, 8 in 2024-09-21T11:56:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:47 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: osdmap e145: 8 total, 8 up, 8 in 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2478488293' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-77452-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: pgmap v164: 299 pgs: 32 unknown, 8 creating+peering, 259 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:48.591 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: osdmap e145: 8 total, 8 up, 8 in 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2478488293' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-77452-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: pgmap v164: 299 pgs: 32 unknown, 8 creating+peering, 259 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-77431-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: osdmap e145: 8 total, 8 up, 8 in 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2478488293' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-77452-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: pgmap v164: 299 pgs: 32 unknown, 8 creating+peering, 259 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:56:49.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:56:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:56:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2478488293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-77452-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: osdmap e146: 8 total, 8 up, 8 in 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: osdmap e147: 8 total, 8 up, 8 in 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2478488293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-77452-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: osdmap e146: 8 total, 8 up, 8 in 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: osdmap e147: 8 total, 8 up, 8 in 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2478488293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-77452-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: osdmap e146: 8 total, 8 up, 8 in 2024-09-21T11:56:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-77431-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: osdmap e147: 8 total, 8 up, 8 in 2024-09-21T11:56:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"} : dispatch 2024-09-21T11:56:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:50.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[30293]: pgmap v167: 275 pgs: 8 creating+peering, 8 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[30293]: osdmap e148: 8 total, 8 up, 8 in 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3596154198' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[41097]: pgmap v167: 275 pgs: 8 creating+peering, 8 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[41097]: osdmap e148: 8 total, 8 up, 8 in 2024-09-21T11:56:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3596154198' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:50 smithi193 ceph-mon[33333]: pgmap v167: 275 pgs: 8 creating+peering, 8 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1280464482' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77697-24"}]': finished 2024-09-21T11:56:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:50 smithi193 ceph-mon[33333]: osdmap e148: 8 total, 8 up, 8 in 2024-09-21T11:56:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3596154198' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:56:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: osdmap e149: 8 total, 8 up, 8 in 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1753102356' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-77697-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: pgmap v170: 331 pgs: 8 creating+peering, 64 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: osdmap e149: 8 total, 8 up, 8 in 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1753102356' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-77697-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: pgmap v170: 331 pgs: 8 creating+peering, 64 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:52 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-77452-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: osdmap e149: 8 total, 8 up, 8 in 2024-09-21T11:56:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1753102356' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-77697-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: pgmap v170: 331 pgs: 8 creating+peering, 64 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:52 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:53.331 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: Running main() from gmock_main.cc 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] Global test environment set-up. 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2184 ms) 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3047 ms) 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (2994 ms) 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3005 ms) 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-21T11:56:53.332 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (2416 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2984 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3040 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2957 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3030 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2987 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3016 ms) 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-21T11:56:53.333 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3021 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2963 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3000 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.Flush (2992 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3010 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2988 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2996 ms) 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-21T11:56:53.334 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3019 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (2999 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3012 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3015 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2956 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3025 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2984 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3004 ms) 2024-09-21T11:56:53.335 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 26 tests from LibRadosAio (76644 ms total) 2024-09-21T11:56:53.336 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: 2024-09-21T11:56:53.336 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-21T11:56:53.336 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-21T11:56:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1753102356' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-77697-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: osdmap e150: 8 total, 8 up, 8 in 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1753102356' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-77697-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: osdmap e150: 8 total, 8 up, 8 in 2024-09-21T11:56:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:53.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:53.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:53.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:53.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1753102356' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-77697-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: osdmap e150: 8 total, 8 up, 8 in 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3297929969' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"} : dispatch 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:53.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-77431-27"}]': finished 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: osdmap e151: 8 total, 8 up, 8 in 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3375225846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: pgmap v173: 299 pgs: 8 creating+peering, 32 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-77431-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-77431-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: osdmap e152: 8 total, 8 up, 8 in 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-77431-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13"} : dispatch 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/170963346' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-77697-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: osdmap e151: 8 total, 8 up, 8 in 2024-09-21T11:56:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3375225846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: pgmap v173: 299 pgs: 8 creating+peering, 32 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-77431-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-77431-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: osdmap e152: 8 total, 8 up, 8 in 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-77431-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13"} : dispatch 2024-09-21T11:56:54.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/170963346' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-77697-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: osdmap e151: 8 total, 8 up, 8 in 2024-09-21T11:56:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3375225846' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: pgmap v173: 299 pgs: 8 creating+peering, 32 unknown, 259 active+clean; 592 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-77431-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-77452-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-77431-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: osdmap e152: 8 total, 8 up, 8 in 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-77431-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13"} : dispatch 2024-09-21T11:56:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/170963346' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-77697-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13"}]': finished 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/170963346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-77697-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[30293]: osdmap e153: 8 total, 8 up, 8 in 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13"}]': finished 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/170963346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-77697-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[41097]: osdmap e153: 8 total, 8 up, 8 in 2024-09-21T11:56:55.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:55 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:56:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-13"}]': finished 2024-09-21T11:56:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/170963346' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-77697-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:55 smithi193 ceph-mon[33333]: osdmap e153: 8 total, 8 up, 8 in 2024-09-21T11:56:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[30293]: pgmap v176: 299 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 32 unknown, 249 active+clean; 587 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-77431-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[30293]: osdmap e154: 8 total, 8 up, 8 in 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/354432948' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[41097]: pgmap v176: 299 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 32 unknown, 249 active+clean; 587 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-09-21T11:56:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-77431-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:56:56.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[41097]: osdmap e154: 8 total, 8 up, 8 in 2024-09-21T11:56:56.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/354432948' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:56.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:56.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:56 smithi193 ceph-mon[33333]: pgmap v176: 299 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 32 unknown, 249 active+clean; 587 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 7 op/s 2024-09-21T11:56:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-77431-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:56:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:56 smithi193 ceph-mon[33333]: osdmap e154: 8 total, 8 up, 8 in 2024-09-21T11:56:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/354432948' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: osdmap e155: 8 total, 8 up, 8 in 2024-09-21T11:56:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/177869339' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:57.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:57.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:57.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: osdmap e155: 8 total, 8 up, 8 in 2024-09-21T11:56:57.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/177869339' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-77452-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:57.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: osdmap e155: 8 total, 8 up, 8 in 2024-09-21T11:56:57.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/177869339' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-15","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:57.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[30293]: pgmap v179: 339 pgs: 1 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 104 unknown, 225 active+clean; 586 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[30293]: osdmap e156: 8 total, 8 up, 8 in 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[41097]: pgmap v179: 339 pgs: 1 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 104 unknown, 225 active+clean; 586 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-09-21T11:56:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[41097]: osdmap e156: 8 total, 8 up, 8 in 2024-09-21T11:56:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:58.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:56:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-mon[33333]: pgmap v179: 339 pgs: 1 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 104 unknown, 225 active+clean; 586 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-09-21T11:56:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-77697-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:58.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-mon[33333]: osdmap e156: 8 total, 8 up, 8 in 2024-09-21T11:56:58.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:58.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: osdmap e157: 8 total, 8 up, 8 in 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3410614731' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: osdmap e157: 8 total, 8 up, 8 in 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3410614731' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:59.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:59.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:56:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:56:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:56:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: osdmap e157: 8 total, 8 up, 8 in 2024-09-21T11:56:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3410614731' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:56:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:56:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: pgmap v182: 299 pgs: 32 unknown, 267 active+clean; 618 KiB data, 569 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:57:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: osdmap e158: 8 total, 8 up, 8 in 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-15"} : dispatch 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:00.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:00.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:00.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: pgmap v182: 299 pgs: 32 unknown, 267 active+clean; 618 KiB data, 569 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:00.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:57:00.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:00.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:00.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: osdmap e158: 8 total, 8 up, 8 in 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-15"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:00.759 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:00.759 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: pgmap v182: 299 pgs: 32 unknown, 267 active+clean; 618 KiB data, 569 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:00.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2451021560' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-77431-28"}]': finished 2024-09-21T11:57:00.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-77452-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:00.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:00.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:00.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: osdmap e158: 8 total, 8 up, 8 in 2024-09-21T11:57:00.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-15"} : dispatch 2024-09-21T11:57:00.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:00.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:00.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:00.803 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:00.803 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:00.803 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-15"}]': finished 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: osdmap e159: 8 total, 8 up, 8 in 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-15", "mode": "writeback"} : dispatch 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.603 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: pgmap v185: 259 pgs: 259 active+clean; 586 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-15", "mode": "writeback"}]': finished 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.604 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:02 smithi193 ceph-mon[33333]: osdmap e160: 8 total, 8 up, 8 in 2024-09-21T11:57:02.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:57:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-15"}]': finished 2024-09-21T11:57:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: osdmap e159: 8 total, 8 up, 8 in 2024-09-21T11:57:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-15", "mode": "writeback"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: pgmap v185: 259 pgs: 259 active+clean; 586 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-15", "mode": "writeback"}]': finished 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[30293]: osdmap e160: 8 total, 8 up, 8 in 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-15"}]': finished 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-77431-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: osdmap e159: 8 total, 8 up, 8 in 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-15", "mode": "writeback"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: pgmap v185: 259 pgs: 259 active+clean; 586 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"} : dispatch 2024-09-21T11:57:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-77697-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-15", "mode": "writeback"}]': finished 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-77529-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3718008329' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:02.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:02 smithi107 ceph-mon[41097]: osdmap e160: 8 total, 8 up, 8 in 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: SnapshotsECPP.SnapRemovePP (1957 ms) 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2024 ms) 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1978 ms) 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8970 ms total) 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-21T11:57:03.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4044 ms) 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3982 ms) 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3965 ms) 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11993 ms total) 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (93514 ms total) 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-09-21T11:57:03.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1879073452' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:03.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[41097]: osdmap e161: 8 total, 8 up, 8 in 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1879073452' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:03.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:03 smithi107 ceph-mon[30293]: osdmap e161: 8 total, 8 up, 8 in 2024-09-21T11:57:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1879073452' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"} : dispatch 2024-09-21T11:57:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3380087174' entity='client.admin' cmd={"prefix":"status","format":"json"} : dispatch 2024-09-21T11:57:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-77431-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-77822-21"}]': finished 2024-09-21T11:57:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-77452-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:03 smithi193 ceph-mon[33333]: osdmap e161: 8 total, 8 up, 8 in 2024-09-21T11:57:04.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: pgmap v188: 307 pgs: 48 unknown, 259 active+clean; 586 KiB data, 569 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: osdmap e162: 8 total, 8 up, 8 in 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15"} : dispatch 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: pgmap v188: 307 pgs: 48 unknown, 259 active+clean; 586 KiB data, 569 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: osdmap e162: 8 total, 8 up, 8 in 2024-09-21T11:57:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15"} : dispatch 2024-09-21T11:57:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: pgmap v188: 307 pgs: 48 unknown, 259 active+clean; 586 KiB data, 569 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: osdmap e162: 8 total, 8 up, 8 in 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15"} : dispatch 2024-09-21T11:57:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15"}]': finished 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: osdmap e163: 8 total, 8 up, 8 in 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1689813699' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-77452-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15"}]': finished 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: osdmap e163: 8 total, 8 up, 8 in 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1689813699' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-77452-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:05.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-15"}]': finished 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-77529-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: osdmap e163: 8 total, 8 up, 8 in 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1677024376' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"} : dispatch 2024-09-21T11:57:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:05.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:05.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1689813699' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-77452-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:05.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:06.293 INFO:tasks.workunit.client.0.smithi107.stdout:(67485 ms total) 2024-09-21T11:57:06.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:57:06.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-21T11:57:06.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-21T11:57:06.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (115 ms) 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (115 ms total) 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (21 ms) 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (22 ms) 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (43 ms total) 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (23 ms) 2024-09-21T11:57:06.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (8 ms) 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (31 ms total) 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (49 ms) 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (4 ms) 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (53 ms total) 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1032 ms) 2024-09-21T11:57:06.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1032 ms total) 2024-09-21T11:57:06.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-21T11:57:06.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-21T11:57:06.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (96538 ms total) 2024-09-21T11:57:06.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-21T11:57:06.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: pgmap v191: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1689813699' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-77452-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: osdmap e164: 8 total, 8 up, 8 in 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: pgmap v191: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1689813699' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-77452-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: osdmap e164: 8 total, 8 up, 8 in 2024-09-21T11:57:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: pgmap v191: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-77697-36"}]': finished 2024-09-21T11:57:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1689813699' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-77452-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1703661315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: osdmap e164: 8 total, 8 up, 8 in 2024-09-21T11:57:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"} : dispatch 2024-09-21T11:57:07.282 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2174 ms) 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (17482 ms) 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (5993 ms) 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-21T11:57:07.283 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3007 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (2995 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3025 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2985 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2980 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2990 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3003 ms) 2024-09-21T11:57:07.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2995 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2999 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3009 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3012 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3008 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3006 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2964 ms) 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-21T11:57:07.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3023 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (2992 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2997 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3013 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3014 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3019 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2951 ms) 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-21T11:57:07.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3022 ms) 2024-09-21T11:57:07.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-21T11:57:07.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2980 ms) 2024-09-21T11:57:07.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-21T11:57:07.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: osdmap e165: 8 total, 8 up, 8 in 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-77431-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:07.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: osdmap e165: 8 total, 8 up, 8 in 2024-09-21T11:57:07.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:07.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:07.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:07.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-77431-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-77529-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-77431-29"}]': finished 2024-09-21T11:57:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: osdmap e165: 8 total, 8 up, 8 in 2024-09-21T11:57:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-17","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:07.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:07.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-77431-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:08.354 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioexpected=1 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:de5d7c5f:::12:head expected=13:de5d7c5f:::12:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=12 expected=12 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:d83876eb:::4:head expected=13:d83876eb:::4:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=4 expected=4 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:cfc208b3:::3:head expected=13:cfc208b3:::3:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=3 expected=3 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:566253c9:::13:head 2024-09-21T11:57:08.355 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:566253c9:::13:head expected=13:566253c9:::13:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=13 expected=13 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:863748b0:::15:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:863748b0:::15:head expected=13:863748b0:::15:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=15 expected=15 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:6cac518f:::0:head expected=13:6cac518f:::0:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=0 expected=0 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:5c6b0b28:::7:head expected=13:5c6b0b28:::7:head 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-09-21T11:57:08.356 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=7 expected=7 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=13:62a1935d:::14:head expected=13:62a1935d:::14:head 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=14 expected=14 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (169 ms) 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (50042 ms) 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 0/5 -> MIN 13:33333333::::head 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 1/5 -> 13:33333333::::head 13:66666666::::head 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 2/5 -> 13:66666666::::head 13:99999999::::head 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 3/5 -> 13:99999999::::head 13:cccccccc::::head 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 4/5 -> 13:cccccccc::::head MAX 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (42002 ms) 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 7 tests from LibRadosList (92548 ms total) 2024-09-21T11:57:08.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1044 ms) 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo2,foo3 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo2 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo3 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo4,foo5 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo4 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo5 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo6,foo7 2024-09-21T11:57:08.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo7 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo6 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo4 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo5 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo7 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo6 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo1 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo1 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo2 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo3 2024-09-21T11:57:08.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (29 ms) 2024-09-21T11:57:09.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: pgmap v194: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-77431-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: osdmap e166: 8 total, 8 up, 8 in 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-77431-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/366252335' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:09.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: pgmap v194: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-77431-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: osdmap e166: 8 total, 8 up, 8 in 2024-09-21T11:57:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-77431-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/366252335' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: pgmap v194: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-77431-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: osdmap e166: 8 total, 8 up, 8 in 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-77431-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/366252335' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:09.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:09.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: osdmap e167: 8 total, 8 up, 8 in 2024-09-21T11:57:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-17"} : dispatch 2024-09-21T11:57:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: osdmap e167: 8 total, 8 up, 8 in 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-17"} : dispatch 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:10.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-77452-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: osdmap e167: 8 total, 8 up, 8 in 2024-09-21T11:57:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-17"} : dispatch 2024-09-21T11:57:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:10.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: pgmap v197: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-77431-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-17"}]': finished 2024-09-21T11:57:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: osdmap e168: 8 total, 8 up, 8 in 2024-09-21T11:57:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-17", "mode": "writeback"} : dispatch 2024-09-21T11:57:11.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: pgmap v197: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-77431-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-17"}]': finished 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: osdmap e168: 8 total, 8 up, 8 in 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-17", "mode": "writeback"} : dispatch 2024-09-21T11:57:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: pgmap v197: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-77431-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-17"}]': finished 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: osdmap e168: 8 total, 8 up, 8 in 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1727307081' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-17", "mode": "writeback"} : dispatch 2024-09-21T11:57:11.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"} : dispatch 2024-09-21T11:57:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-17", "mode": "writeback"}]': finished 2024-09-21T11:57:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: osdmap e169: 8 total, 8 up, 8 in 2024-09-21T11:57:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2773970114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:12.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-17", "mode": "writeback"}]': finished 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: osdmap e169: 8 total, 8 up, 8 in 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2773970114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-17", "mode": "writeback"}]': finished 2024-09-21T11:57:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-77529-2"}]': finished 2024-09-21T11:57:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: osdmap e169: 8 total, 8 up, 8 in 2024-09-21T11:57:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2773970114' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:12.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:57:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: pgmap v200: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: osdmap e170: 8 total, 8 up, 8 in 2024-09-21T11:57:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17"} : dispatch 2024-09-21T11:57:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77529-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: pgmap v200: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: osdmap e170: 8 total, 8 up, 8 in 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17"} : dispatch 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77529-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: pgmap v200: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:13.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-77452-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:13.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:13.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: osdmap e170: 8 total, 8 up, 8 in 2024-09-21T11:57:13.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17"} : dispatch 2024-09-21T11:57:13.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:13.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77529-3","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:14.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17"}]': finished 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77529-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: osdmap e171: 8 total, 8 up, 8 in 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi107-77529-3","pool2":"test-rados-api-smithi107-77529-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17"}]': finished 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77529-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:14.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: osdmap e171: 8 total, 8 up, 8 in 2024-09-21T11:57:14.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:14.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi107-77529-3","pool2":"test-rados-api-smithi107-77529-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-17"}]': finished 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77529-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: osdmap e171: 8 total, 8 up, 8 in 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"} : dispatch 2024-09-21T11:57:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd={"prefix":"osd pool rm","pool": "test-rados-api-smithi107-77529-3","pool2":"test-rados-api-smithi107-77529-3","yes_i_really_really_mean_it_not_faking": true} : dispatch 2024-09-21T11:57:14.348 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 1 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 10 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 13 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 7 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 14 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 0 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 15 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 11 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 5 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 8 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 6 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 3 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 4 0 2024-09-21T11:57:14.349 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 12 0 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 9 0 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2 0 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (42 ms) 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 3 tests from LibRadosListEC (1116 ms total) 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3035 ms) 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 1 test from LibRadosListNP (3035 ms total) 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] Global test environment tear-down 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [==========] 11 tests from 3 test suites ran. (104663 ms total) 2024-09-21T11:57:14.350 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ PASSED ] 11 tests. 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: pgmap v203: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-77529-3","pool2":"test-rados-api-smithi107-77529-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: osdmap e172: 8 total, 8 up, 8 in 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3075990711' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: pgmap v203: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-77529-3","pool2":"test-rados-api-smithi107-77529-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: osdmap e172: 8 total, 8 up, 8 in 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3075990711' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:15.256 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: pgmap v203: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4289146127' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-77431-30"}]': finished 2024-09-21T11:57:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2426609363' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-77529-3","pool2":"test-rados-api-smithi107-77529-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: osdmap e172: 8 total, 8 up, 8 in 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3075990711' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:15.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: osdmap e173: 8 total, 8 up, 8 in 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: pgmap v206: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: osdmap e173: 8 total, 8 up, 8 in 2024-09-21T11:57:16.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: pgmap v206: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-77452-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-77431-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: osdmap e173: 8 total, 8 up, 8 in 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-19","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: pgmap v206: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:17.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[30293]: osdmap e174: 8 total, 8 up, 8 in 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[41097]: osdmap e174: 8 total, 8 up, 8 in 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:17.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:17 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:17 smithi193 ceph-mon[33333]: osdmap e174: 8 total, 8 up, 8 in 2024-09-21T11:57:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:17 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:18.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-mon[33333]: pgmap v208: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-21T11:57:18.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:18.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:18.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-mon[33333]: osdmap e175: 8 total, 8 up, 8 in 2024-09-21T11:57:18.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-19"} : dispatch 2024-09-21T11:57:18.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-77452-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:18.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[30293]: pgmap v208: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-21T11:57:18.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:18.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[30293]: osdmap e175: 8 total, 8 up, 8 in 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-19"} : dispatch 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-77452-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[41097]: pgmap v208: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-77431-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[41097]: osdmap e175: 8 total, 8 up, 8 in 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-19"} : dispatch 2024-09-21T11:57:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-77452-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:19.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:57:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-19"}]': finished 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-77452-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: osdmap e176: 8 total, 8 up, 8 in 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-19", "mode": "writeback"} : dispatch 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd={ 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "prefix": "osd pool set", 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "pool": "PoolEIOFlag_smithi107-77452-33", 2024-09-21T11:57:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "var": "eio", 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "val": "true" 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: } : dispatch 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-19", "mode": "writeback"}]': finished 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd='[{ 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "prefix": "osd pool set", 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "pool": "PoolEIOFlag_smithi107-77452-33", 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "var": "eio", 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: "val": "true" 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: }]': finished 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[30293]: osdmap e177: 8 total, 8 up, 8 in 2024-09-21T11:57:19.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-19"}]': finished 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-77452-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: osdmap e176: 8 total, 8 up, 8 in 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-19", "mode": "writeback"} : dispatch 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd={ 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "prefix": "osd pool set", 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "pool": "PoolEIOFlag_smithi107-77452-33", 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "var": "eio", 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "val": "true" 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: } : dispatch 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:57:19.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-19", "mode": "writeback"}]': finished 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd='[{ 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "prefix": "osd pool set", 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "pool": "PoolEIOFlag_smithi107-77452-33", 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "var": "eio", 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: "val": "true" 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: }]': finished 2024-09-21T11:57:19.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:19 smithi107 ceph-mon[41097]: osdmap e177: 8 total, 8 up, 8 in 2024-09-21T11:57:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-19"}]': finished 2024-09-21T11:57:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-77452-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: osdmap e176: 8 total, 8 up, 8 in 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-19", "mode": "writeback"} : dispatch 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd={ 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "prefix": "osd pool set", 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "pool": "PoolEIOFlag_smithi107-77452-33", 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "var": "eio", 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "val": "true" 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: } : dispatch 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-19", "mode": "writeback"}]': finished 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2867665107' entity='client.admin' cmd='[{ 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "prefix": "osd pool set", 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "pool": "PoolEIOFlag_smithi107-77452-33", 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "var": "eio", 2024-09-21T11:57:19.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: "val": "true" 2024-09-21T11:57:19.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: }]': finished 2024-09-21T11:57:19.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:19 smithi193 ceph-mon[33333]: osdmap e177: 8 total, 8 up, 8 in 2024-09-21T11:57:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: pgmap v211: 268 pgs: 16 creating+peering, 252 active+clean; 596 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-09-21T11:57:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: pgmap v211: 268 pgs: 16 creating+peering, 252 active+clean; 596 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: pgmap v211: 268 pgs: 16 creating+peering, 252 active+clean; 596 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: osdmap e178: 8 total, 8 up, 8 in 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19"} : dispatch 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19"}]': finished 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: osdmap e179: 8 total, 8 up, 8 in 2024-09-21T11:57:21.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4203617298' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi107-77452-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: osdmap e178: 8 total, 8 up, 8 in 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19"} : dispatch 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19"}]': finished 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: osdmap e179: 8 total, 8 up, 8 in 2024-09-21T11:57:21.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4203617298' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi107-77452-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:21.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:21.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: osdmap e178: 8 total, 8 up, 8 in 2024-09-21T11:57:21.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2906237963' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19"} : dispatch 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"} : dispatch 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-19"}]': finished 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-77431-31"}]': finished 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: osdmap e179: 8 total, 8 up, 8 in 2024-09-21T11:57:21.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4203617298' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "MultiReads_smithi107-77452-34","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:57:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: pgmap v214: 228 pgs: 228 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4203617298' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-77452-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: osdmap e180: 8 total, 8 up, 8 in 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: pgmap v214: 228 pgs: 228 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:22.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4203617298' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-77452-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: osdmap e180: 8 total, 8 up, 8 in 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: pgmap v214: 228 pgs: 228 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4203617298' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-77452-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-77431-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: osdmap e180: 8 total, 8 up, 8 in 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:22.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:23 smithi107 ceph-mon[30293]: osdmap e181: 8 total, 8 up, 8 in 2024-09-21T11:57:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:23.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:23.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:23 smithi107 ceph-mon[41097]: osdmap e181: 8 total, 8 up, 8 in 2024-09-21T11:57:23.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:23 smithi193 ceph-mon[33333]: osdmap e181: 8 total, 8 up, 8 in 2024-09-21T11:57:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-21","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[30293]: pgmap v217: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[30293]: osdmap e182: 8 total, 8 up, 8 in 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1176580813' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[41097]: pgmap v217: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[41097]: osdmap e182: 8 total, 8 up, 8 in 2024-09-21T11:57:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1176580813' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:24 smithi193 ceph-mon[33333]: pgmap v217: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-77431-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:24 smithi193 ceph-mon[33333]: osdmap e182: 8 total, 8 up, 8 in 2024-09-21T11:57:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1176580813' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[30293]: osdmap e183: 8 total, 8 up, 8 in 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-21"} : dispatch 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[41097]: osdmap e183: 8 total, 8 up, 8 in 2024-09-21T11:57:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-21"} : dispatch 2024-09-21T11:57:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-77452-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:25 smithi193 ceph-mon[33333]: osdmap e183: 8 total, 8 up, 8 in 2024-09-21T11:57:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-21"} : dispatch 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[30293]: pgmap v220: 268 pgs: 12 creating+peering, 60 unknown, 196 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-21"}]': finished 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[30293]: osdmap e184: 8 total, 8 up, 8 in 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-21", "mode": "writeback"} : dispatch 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[41097]: pgmap v220: 268 pgs: 12 creating+peering, 60 unknown, 196 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-21"}]': finished 2024-09-21T11:57:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[41097]: osdmap e184: 8 total, 8 up, 8 in 2024-09-21T11:57:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-21", "mode": "writeback"} : dispatch 2024-09-21T11:57:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:26 smithi193 ceph-mon[33333]: pgmap v220: 268 pgs: 12 creating+peering, 60 unknown, 196 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-21"}]': finished 2024-09-21T11:57:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:26 smithi193 ceph-mon[33333]: osdmap e184: 8 total, 8 up, 8 in 2024-09-21T11:57:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-21", "mode": "writeback"} : dispatch 2024-09-21T11:57:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:27.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-21", "mode": "writeback"}]': finished 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: osdmap e185: 8 total, 8 up, 8 in 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3732087161' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-21", "mode": "writeback"}]': finished 2024-09-21T11:57:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: osdmap e185: 8 total, 8 up, 8 in 2024-09-21T11:57:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3732087161' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-21", "mode": "writeback"}]': finished 2024-09-21T11:57:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: osdmap e185: 8 total, 8 up, 8 in 2024-09-21T11:57:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3732087161' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-36","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:27.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2141919924' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:27.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"} : dispatch 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: pgmap v223: 228 pgs: 12 creating+peering, 20 unknown, 196 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3732087161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: osdmap e186: 8 total, 8 up, 8 in 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21"} : dispatch 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: pgmap v223: 228 pgs: 12 creating+peering, 20 unknown, 196 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3732087161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: osdmap e186: 8 total, 8 up, 8 in 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21"} : dispatch 2024-09-21T11:57:28.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:28.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:28 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:28.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:57:28.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: pgmap v223: 228 pgs: 12 creating+peering, 20 unknown, 196 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3732087161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-77431-32"}]': finished 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: osdmap e186: 8 total, 8 up, 8 in 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21"} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:28.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:28 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:29.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21"}]': finished 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: osdmap e187: 8 total, 8 up, 8 in 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3585270506' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21"}]': finished 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: osdmap e187: 8 total, 8 up, 8 in 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3585270506' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-21"}]': finished 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-77431-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: osdmap e187: 8 total, 8 up, 8 in 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3585270506' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:29.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:30 smithi107 ceph-mon[30293]: pgmap v226: 260 pgs: 14 creating+peering, 246 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T11:57:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:30 smithi107 ceph-mon[30293]: osdmap e188: 8 total, 8 up, 8 in 2024-09-21T11:57:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:30 smithi107 ceph-mon[41097]: pgmap v226: 260 pgs: 14 creating+peering, 246 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T11:57:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:30 smithi107 ceph-mon[41097]: osdmap e188: 8 total, 8 up, 8 in 2024-09-21T11:57:30.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:30 smithi193 ceph-mon[33333]: pgmap v226: 260 pgs: 14 creating+peering, 246 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T11:57:30.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:30.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:30 smithi193 ceph-mon[33333]: osdmap e188: 8 total, 8 up, 8 in 2024-09-21T11:57:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:57:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: pgmap v229: 260 pgs: 32 unknown, 14 creating+peering, 214 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: osdmap e189: 8 total, 8 up, 8 in 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1491281711' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: pgmap v229: 260 pgs: 32 unknown, 14 creating+peering, 214 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: osdmap e189: 8 total, 8 up, 8 in 2024-09-21T11:57:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1491281711' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:32 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: pgmap v229: 260 pgs: 32 unknown, 14 creating+peering, 214 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-77431-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: osdmap e189: 8 total, 8 up, 8 in 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-23","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1491281711' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:32 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[30293]: osdmap e190: 8 total, 8 up, 8 in 2024-09-21T11:57:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[30293]: osdmap e191: 8 total, 8 up, 8 in 2024-09-21T11:57:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:33.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:33.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:33.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[41097]: osdmap e190: 8 total, 8 up, 8 in 2024-09-21T11:57:33.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[41097]: osdmap e191: 8 total, 8 up, 8 in 2024-09-21T11:57:33.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:33.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-77452-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:33 smithi193 ceph-mon[33333]: osdmap e190: 8 total, 8 up, 8 in 2024-09-21T11:57:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:33 smithi193 ceph-mon[33333]: osdmap e191: 8 total, 8 up, 8 in 2024-09-21T11:57:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: pgmap v232: 332 pgs: 104 unknown, 14 creating+peering, 214 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: osdmap e192: 8 total, 8 up, 8 in 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-23"} : dispatch 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: pgmap v232: 332 pgs: 104 unknown, 14 creating+peering, 214 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: osdmap e192: 8 total, 8 up, 8 in 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-23"} : dispatch 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: pgmap v232: 332 pgs: 104 unknown, 14 creating+peering, 214 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: osdmap e192: 8 total, 8 up, 8 in 2024-09-21T11:57:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-23"} : dispatch 2024-09-21T11:57:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4235318958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: pgmap v235: 260 pgs: 260 active+clean; 584 KiB data, 550 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-23"}]': finished 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: osdmap e193: 8 total, 8 up, 8 in 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-23", "mode": "writeback"} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: pgmap v235: 260 pgs: 260 active+clean; 584 KiB data, 550 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-23"}]': finished 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: osdmap e193: 8 total, 8 up, 8 in 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-23", "mode": "writeback"} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:36.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: pgmap v235: 260 pgs: 260 active+clean; 584 KiB data, 550 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:57:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-23"}]': finished 2024-09-21T11:57:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-77431-33"}]': finished 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: osdmap e193: 8 total, 8 up, 8 in 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-23", "mode": "writeback"} : dispatch 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:37.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-23", "mode": "writeback"}]': finished 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: osdmap e194: 8 total, 8 up, 8 in 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3944778265' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3944778265' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[30293]: osdmap e195: 8 total, 8 up, 8 in 2024-09-21T11:57:37.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-23", "mode": "writeback"}]': finished 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: osdmap e194: 8 total, 8 up, 8 in 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3944778265' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3944778265' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:37.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:37 smithi107 ceph-mon[41097]: osdmap e195: 8 total, 8 up, 8 in 2024-09-21T11:57:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-23", "mode": "writeback"}]': finished 2024-09-21T11:57:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-77431-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: osdmap e194: 8 total, 8 up, 8 in 2024-09-21T11:57:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3944778265' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3944778265' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:37.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:37 smithi193 ceph-mon[33333]: osdmap e195: 8 total, 8 up, 8 in 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: pgmap v238: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 550 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: osdmap e196: 8 total, 8 up, 8 in 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23"} : dispatch 2024-09-21T11:57:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1857032570' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: pgmap v238: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 550 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: osdmap e196: 8 total, 8 up, 8 in 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23"} : dispatch 2024-09-21T11:57:38.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1857032570' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:38.801 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:57:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: pgmap v238: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 550 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-77431-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: osdmap e196: 8 total, 8 up, 8 in 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23"} : dispatch 2024-09-21T11:57:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1857032570' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-40","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23"}]': finished 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1857032570' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[41097]: osdmap e197: 8 total, 8 up, 8 in 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23"}]': finished 2024-09-21T11:57:39.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1857032570' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:39.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:39 smithi107 ceph-mon[30293]: osdmap e197: 8 total, 8 up, 8 in 2024-09-21T11:57:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:39 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-23"}]': finished 2024-09-21T11:57:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1857032570' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:39 smithi193 ceph-mon[33333]: osdmap e197: 8 total, 8 up, 8 in 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[30293]: pgmap v241: 300 pgs: 15 creating+peering, 25 unknown, 260 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[30293]: osdmap e198: 8 total, 8 up, 8 in 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/488712923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[41097]: pgmap v241: 300 pgs: 15 creating+peering, 25 unknown, 260 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[41097]: osdmap e198: 8 total, 8 up, 8 in 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/488712923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:40 smithi193 ceph-mon[33333]: pgmap v241: 300 pgs: 15 creating+peering, 25 unknown, 260 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-21T11:57:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:40 smithi193 ceph-mon[33333]: osdmap e198: 8 total, 8 up, 8 in 2024-09-21T11:57:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/488712923' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:40.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:57:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: pgmap v244: 292 pgs: 11 creating+peering, 53 unknown, 228 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/488712923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: osdmap e199: 8 total, 8 up, 8 in 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: osdmap e200: 8 total, 8 up, 8 in 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3024844299' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-77431-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: pgmap v244: 292 pgs: 11 creating+peering, 53 unknown, 228 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T11:57:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/488712923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: osdmap e199: 8 total, 8 up, 8 in 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: osdmap e200: 8 total, 8 up, 8 in 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3024844299' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:42.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-77431-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: pgmap v244: 292 pgs: 11 creating+peering, 53 unknown, 228 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T11:57:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/488712923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: osdmap e199: 8 total, 8 up, 8 in 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/16704669' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-25","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-77431-34"}]': finished 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: osdmap e200: 8 total, 8 up, 8 in 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3024844299' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-77431-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:44 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-77431-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:44 smithi193 ceph-mon[33333]: osdmap e201: 8 total, 8 up, 8 in 2024-09-21T11:57:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi107-77431-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:44.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:44 smithi193 ceph-mon[33333]: pgmap v248: 356 pgs: 11 creating+peering, 99 unknown, 246 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:44.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-77431-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[30293]: osdmap e201: 8 total, 8 up, 8 in 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi107-77431-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[30293]: pgmap v248: 356 pgs: 11 creating+peering, 99 unknown, 246 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-77431-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[41097]: osdmap e201: 8 total, 8 up, 8 in 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "Flush_smithi107-77431-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:44 smithi107 ceph-mon[41097]: pgmap v248: 356 pgs: 11 creating+peering, 99 unknown, 246 active+clean; 584 KiB data, 551 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:45 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:45 smithi193 ceph-mon[33333]: osdmap e202: 8 total, 8 up, 8 in 2024-09-21T11:57:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-25"} : dispatch 2024-09-21T11:57:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/149576069' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:45.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[41097]: osdmap e202: 8 total, 8 up, 8 in 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-25"} : dispatch 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/149576069' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[30293]: osdmap e202: 8 total, 8 up, 8 in 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-25"} : dispatch 2024-09-21T11:57:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/149576069' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi107-77431-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-25"}]': finished 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/149576069' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[30293]: osdmap e203: 8 total, 8 up, 8 in 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-25", "mode": "writeback"} : dispatch 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[30293]: pgmap v251: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 779 B/s wr, 2 op/s 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi107-77431-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-25"}]': finished 2024-09-21T11:57:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/149576069' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[41097]: osdmap e203: 8 total, 8 up, 8 in 2024-09-21T11:57:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-25", "mode": "writeback"} : dispatch 2024-09-21T11:57:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:46 smithi107 ceph-mon[41097]: pgmap v251: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 779 B/s wr, 2 op/s 2024-09-21T11:57:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi107-77431-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-25"}]': finished 2024-09-21T11:57:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/149576069' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-77452-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:46 smithi193 ceph-mon[33333]: osdmap e203: 8 total, 8 up, 8 in 2024-09-21T11:57:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-25", "mode": "writeback"} : dispatch 2024-09-21T11:57:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:46 smithi193 ceph-mon[33333]: pgmap v251: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 779 B/s wr, 2 op/s 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-25", "mode": "writeback"}]': finished 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[30293]: osdmap e204: 8 total, 8 up, 8 in 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-25", "mode": "writeback"}]': finished 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[41097]: osdmap e204: 8 total, 8 up, 8 in 2024-09-21T11:57:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:47 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-25", "mode": "writeback"}]': finished 2024-09-21T11:57:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:47 smithi193 ceph-mon[33333]: osdmap e204: 8 total, 8 up, 8 in 2024-09-21T11:57:47.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:48.633 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:48.633 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:48 smithi193 ceph-mon[33333]: osdmap e205: 8 total, 8 up, 8 in 2024-09-21T11:57:48.634 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25"} : dispatch 2024-09-21T11:57:48.634 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:48.634 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:48 smithi193 ceph-mon[33333]: pgmap v254: 324 pgs: 324 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[30293]: osdmap e205: 8 total, 8 up, 8 in 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25"} : dispatch 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[30293]: pgmap v254: 324 pgs: 324 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[41097]: osdmap e205: 8 total, 8 up, 8 in 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25"} : dispatch 2024-09-21T11:57:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:48 smithi107 ceph-mon[41097]: pgmap v254: 324 pgs: 324 active+clean; 584 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T11:57:49.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:57:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25"}]': finished 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[30293]: osdmap e206: 8 total, 8 up, 8 in 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25"}]': finished 2024-09-21T11:57:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[41097]: osdmap e206: 8 total, 8 up, 8 in 2024-09-21T11:57:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:49 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-25"}]': finished 2024-09-21T11:57:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:49 smithi193 ceph-mon[33333]: osdmap e206: 8 total, 8 up, 8 in 2024-09-21T11:57:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"} : dispatch 2024-09-21T11:57:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:50.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: osdmap e207: 8 total, 8 up, 8 in 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: pgmap v257: 228 pgs: 228 active+clean; 584 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: osdmap e207: 8 total, 8 up, 8 in 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: pgmap v257: 228 pgs: 228 active+clean; 584 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:50.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1368643181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-77431-35"}]': finished 2024-09-21T11:57:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: osdmap e207: 8 total, 8 up, 8 in 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: pgmap v257: 228 pgs: 228 active+clean; 584 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:50.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:51.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[41097]: osdmap e208: 8 total, 8 up, 8 in 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[30293]: osdmap e208: 8 total, 8 up, 8 in 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-77431-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:51 smithi193 ceph-mon[33333]: osdmap e208: 8 total, 8 up, 8 in 2024-09-21T11:57:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:51.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-27","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:51.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:57:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: pgmap v259: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: osdmap e209: 8 total, 8 up, 8 in 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2455508220' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[41097]: osdmap e210: 8 total, 8 up, 8 in 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: pgmap v259: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: osdmap e209: 8 total, 8 up, 8 in 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2455508220' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:52 smithi107 ceph-mon[30293]: osdmap e210: 8 total, 8 up, 8 in 2024-09-21T11:57:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: pgmap v259: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 560 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:57:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: osdmap e209: 8 total, 8 up, 8 in 2024-09-21T11:57:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2455508220' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-77431-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-77452-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:52 smithi193 ceph-mon[33333]: osdmap e210: 8 total, 8 up, 8 in 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[41097]: osdmap e211: 8 total, 8 up, 8 in 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-27"} : dispatch 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:53.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[30293]: osdmap e211: 8 total, 8 up, 8 in 2024-09-21T11:57:53.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-27"} : dispatch 2024-09-21T11:57:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:57:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:53 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:57:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:57:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:53 smithi193 ceph-mon[33333]: osdmap e211: 8 total, 8 up, 8 in 2024-09-21T11:57:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-27"} : dispatch 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: pgmap v262: 268 pgs: 2 creating+peering, 57 unknown, 209 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-27"}]': finished 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: osdmap e212: 8 total, 8 up, 8 in 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-27", "mode": "writeback"} : dispatch 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3411813948' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-77452-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: pgmap v262: 268 pgs: 2 creating+peering, 57 unknown, 209 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:55.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-27"}]': finished 2024-09-21T11:57:55.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: osdmap e212: 8 total, 8 up, 8 in 2024-09-21T11:57:55.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-27", "mode": "writeback"} : dispatch 2024-09-21T11:57:55.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3411813948' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-77452-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:55.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:55.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: pgmap v262: 268 pgs: 2 creating+peering, 57 unknown, 209 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:57:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-27"}]': finished 2024-09-21T11:57:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: osdmap e212: 8 total, 8 up, 8 in 2024-09-21T11:57:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-27", "mode": "writeback"} : dispatch 2024-09-21T11:57:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3411813948' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-77452-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:57:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:55.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:56.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-27", "mode": "writeback"}]': finished 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3411813948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-77452-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: osdmap e213: 8 total, 8 up, 8 in 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-27", "mode": "writeback"}]': finished 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3411813948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-77452-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:56.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: osdmap e213: 8 total, 8 up, 8 in 2024-09-21T11:57:56.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:56.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:57:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-27", "mode": "writeback"}]': finished 2024-09-21T11:57:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3411813948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-77452-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:57:56.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:56.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: osdmap e213: 8 total, 8 up, 8 in 2024-09-21T11:57:56.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3200034340' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:56.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"} : dispatch 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: pgmap v265: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: osdmap e214: 8 total, 8 up, 8 in 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: pgmap v265: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:57:57.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: osdmap e214: 8 total, 8 up, 8 in 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.256 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: pgmap v265: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T11:57:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-77431-36"}]': finished 2024-09-21T11:57:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: osdmap e214: 8 total, 8 up, 8 in 2024-09-21T11:57:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:57:57.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:57:58.638 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: pgmap v268: 228 pgs: 228 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T11:57:58.638 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:58.638 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:58.638 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:58.638 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: osdmap e215: 8 total, 8 up, 8 in 2024-09-21T11:57:58.639 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:58.639 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27"} : dispatch 2024-09-21T11:57:58.639 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:58.639 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:58.639 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:58.639 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: pgmap v268: 228 pgs: 228 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: osdmap e215: 8 total, 8 up, 8 in 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27"} : dispatch 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: pgmap v268: 228 pgs: 228 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: osdmap e215: 8 total, 8 up, 8 in 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27"} : dispatch 2024-09-21T11:57:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:58.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:57:58.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:57:58.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:58 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:57:59.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:57:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:57:59.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27"}]': finished 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: osdmap e216: 8 total, 8 up, 8 in 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[30293]: osdmap e217: 8 total, 8 up, 8 in 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27"}]': finished 2024-09-21T11:57:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: osdmap e216: 8 total, 8 up, 8 in 2024-09-21T11:57:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:57:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:57:59.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:57:59 smithi107 ceph-mon[41097]: osdmap e217: 8 total, 8 up, 8 in 2024-09-21T11:57:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:57:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-27"}]': finished 2024-09-21T11:57:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: osdmap e216: 8 total, 8 up, 8 in 2024-09-21T11:57:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:57:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-77431-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:57:59.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:57:59 smithi193 ceph-mon[33333]: osdmap e217: 8 total, 8 up, 8 in 2024-09-21T11:58:00.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:00 smithi107 ceph-mon[30293]: pgmap v271: 228 pgs: 228 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:58:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:00 smithi107 ceph-mon[30293]: osdmap e218: 8 total, 8 up, 8 in 2024-09-21T11:58:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:00 smithi107 ceph-mon[41097]: pgmap v271: 228 pgs: 228 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:58:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:00 smithi107 ceph-mon[41097]: osdmap e218: 8 total, 8 up, 8 in 2024-09-21T11:58:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:00 smithi193 ceph-mon[33333]: pgmap v271: 228 pgs: 228 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T11:58:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:00 smithi193 ceph-mon[33333]: osdmap e218: 8 total, 8 up, 8 in 2024-09-21T11:58:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-29","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:02.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:58:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: pgmap v274: 244 pgs: 48 unknown, 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: osdmap e219: 8 total, 8 up, 8 in 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: osdmap e220: 8 total, 8 up, 8 in 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: pgmap v274: 244 pgs: 48 unknown, 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: osdmap e219: 8 total, 8 up, 8 in 2024-09-21T11:58:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: osdmap e220: 8 total, 8 up, 8 in 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: pgmap v274: 244 pgs: 48 unknown, 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: osdmap e219: 8 total, 8 up, 8 in 2024-09-21T11:58:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: osdmap e220: 8 total, 8 up, 8 in 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3058436023' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"} : dispatch 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3015820248' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"} : dispatch 2024-09-21T11:58:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:58:04.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: osdmap e221: 8 total, 8 up, 8 in 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-77431-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:04 smithi193 ceph-mon[33333]: pgmap v278: 228 pgs: 21 unknown, 207 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: osdmap e221: 8 total, 8 up, 8 in 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-77431-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[30293]: pgmap v278: 228 pgs: 21 unknown, 207 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-46"}]': finished 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-77431-37"}]': finished 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: osdmap e221: 8 total, 8 up, 8 in 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:04.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:04.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-77431-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:04.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:04 smithi107 ceph-mon[41097]: pgmap v278: 228 pgs: 21 unknown, 207 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-77431-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: osdmap e222: 8 total, 8 up, 8 in 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi107-77431-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-29"} : dispatch 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-77431-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: osdmap e222: 8 total, 8 up, 8 in 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi107-77431-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-29"} : dispatch 2024-09-21T11:58:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-77431-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-77452-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: osdmap e222: 8 total, 8 up, 8 in 2024-09-21T11:58:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStat_smithi107-77431-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-29"} : dispatch 2024-09-21T11:58:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:06.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-29"}]': finished 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: osdmap e223: 8 total, 8 up, 8 in 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-29", "mode": "writeback"} : dispatch 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: pgmap v281: 228 pgs: 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-77431-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-29", "mode": "writeback"}]': finished 2024-09-21T11:58:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[30293]: osdmap e224: 8 total, 8 up, 8 in 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-29"}]': finished 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: osdmap e223: 8 total, 8 up, 8 in 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-29", "mode": "writeback"} : dispatch 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: pgmap v281: 228 pgs: 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-77431-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-29", "mode": "writeback"}]': finished 2024-09-21T11:58:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:06 smithi107 ceph-mon[41097]: osdmap e224: 8 total, 8 up, 8 in 2024-09-21T11:58:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-29"}]': finished 2024-09-21T11:58:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: osdmap e223: 8 total, 8 up, 8 in 2024-09-21T11:58:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-29", "mode": "writeback"} : dispatch 2024-09-21T11:58:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: pgmap v281: 228 pgs: 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-21T11:58:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-77431-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-77452-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-29", "mode": "writeback"}]': finished 2024-09-21T11:58:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:06 smithi193 ceph-mon[33333]: osdmap e224: 8 total, 8 up, 8 in 2024-09-21T11:58:08.648 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:08 smithi193 ceph-mon[33333]: osdmap e225: 8 total, 8 up, 8 in 2024-09-21T11:58:08.649 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:08 smithi193 ceph-mon[33333]: pgmap v284: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-21T11:58:08.649 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:08.649 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:08.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[41097]: osdmap e225: 8 total, 8 up, 8 in 2024-09-21T11:58:08.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[41097]: pgmap v284: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-21T11:58:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[30293]: osdmap e225: 8 total, 8 up, 8 in 2024-09-21T11:58:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[30293]: pgmap v284: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-21T11:58:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:09.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:58:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[30293]: osdmap e226: 8 total, 8 up, 8 in 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[41097]: osdmap e226: 8 total, 8 up, 8 in 2024-09-21T11:58:09.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:09.656 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:09.656 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:09.656 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:09 smithi193 ceph-mon[33333]: osdmap e226: 8 total, 8 up, 8 in 2024-09-21T11:58:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: osdmap e227: 8 total, 8 up, 8 in 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: pgmap v287: 228 pgs: 228 active+clean; 8.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: osdmap e227: 8 total, 8 up, 8 in 2024-09-21T11:58:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: pgmap v287: 228 pgs: 228 active+clean; 8.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-21T11:58:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: osdmap e227: 8 total, 8 up, 8 in 2024-09-21T11:58:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"} : dispatch 2024-09-21T11:58:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3017408627' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"} : dispatch 2024-09-21T11:58:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: pgmap v287: 228 pgs: 228 active+clean; 8.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-21T11:58:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: osdmap e228: 8 total, 8 up, 8 in 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29"} : dispatch 2024-09-21T11:58:11.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: osdmap e228: 8 total, 8 up, 8 in 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:11 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1177746685' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-77431-38"}]': finished 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-77452-47"}]': finished 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: osdmap e228: 8 total, 8 up, 8 in 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:11.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:11 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:12.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:58:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29"}]': finished 2024-09-21T11:58:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: osdmap e229: 8 total, 8 up, 8 in 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[41097]: pgmap v290: 228 pgs: 228 active+clean; 8.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29"}]': finished 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: osdmap e229: 8 total, 8 up, 8 in 2024-09-21T11:58:12.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:12.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:12.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:12.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:12.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:12 smithi107 ceph-mon[30293]: pgmap v290: 228 pgs: 228 active+clean; 8.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-21T11:58:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-29"}]': finished 2024-09-21T11:58:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-77431-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-77452-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: osdmap e229: 8 total, 8 up, 8 in 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:12.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:12 smithi193 ceph-mon[33333]: pgmap v290: 228 pgs: 228 active+clean; 8.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-21T11:58:13.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:13 smithi107 ceph-mon[30293]: osdmap e230: 8 total, 8 up, 8 in 2024-09-21T11:58:13.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:13 smithi107 ceph-mon[41097]: osdmap e230: 8 total, 8 up, 8 in 2024-09-21T11:58:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:13 smithi193 ceph-mon[33333]: osdmap e230: 8 total, 8 up, 8 in 2024-09-21T11:58:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[30293]: pgmap v292: 196 pgs: 196 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[30293]: osdmap e231: 8 total, 8 up, 8 in 2024-09-21T11:58:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:14.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:14.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[41097]: pgmap v292: 196 pgs: 196 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:14.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[41097]: osdmap e231: 8 total, 8 up, 8 in 2024-09-21T11:58:14.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-77452-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-77431-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:14 smithi193 ceph-mon[33333]: pgmap v292: 196 pgs: 196 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:14 smithi193 ceph-mon[33333]: osdmap e231: 8 total, 8 up, 8 in 2024-09-21T11:58:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-31","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:15 smithi107 ceph-mon[41097]: osdmap e232: 8 total, 8 up, 8 in 2024-09-21T11:58:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:15.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:15.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:15 smithi107 ceph-mon[30293]: osdmap e232: 8 total, 8 up, 8 in 2024-09-21T11:58:15.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:15.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:15.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:15 smithi193 ceph-mon[33333]: osdmap e232: 8 total, 8 up, 8 in 2024-09-21T11:58:15.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: pgmap v295: 244 pgs: 23 creating+peering, 25 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: osdmap e233: 8 total, 8 up, 8 in 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-31"} : dispatch 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: pgmap v295: 244 pgs: 23 creating+peering, 25 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: osdmap e233: 8 total, 8 up, 8 in 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-31"} : dispatch 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: pgmap v295: 244 pgs: 23 creating+peering, 25 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: osdmap e233: 8 total, 8 up, 8 in 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-31"} : dispatch 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-31"}]': finished 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: osdmap e234: 8 total, 8 up, 8 in 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-31", "mode": "writeback"} : dispatch 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-31", "mode": "writeback"}]': finished 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: osdmap e235: 8 total, 8 up, 8 in 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-31"}]': finished 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: osdmap e234: 8 total, 8 up, 8 in 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-31", "mode": "writeback"} : dispatch 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-31", "mode": "writeback"}]': finished 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: osdmap e235: 8 total, 8 up, 8 in 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.757 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.758 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-31"}]': finished 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: osdmap e234: 8 total, 8 up, 8 in 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-31", "mode": "writeback"} : dispatch 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1617935233' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3894376331' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"} : dispatch 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"} : dispatch 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-31", "mode": "writeback"}]': finished 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-77431-39"}]': finished 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-77452-48"}]': finished 2024-09-21T11:58:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: osdmap e235: 8 total, 8 up, 8 in 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:17.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:18.658 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: pgmap v299: 228 pgs: 15 creating+peering, 17 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1024 KiB/s wr, 0 op/s 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: osdmap e236: 8 total, 8 up, 8 in 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31"} : dispatch 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:18.659 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: pgmap v299: 228 pgs: 15 creating+peering, 17 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1024 KiB/s wr, 0 op/s 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: osdmap e236: 8 total, 8 up, 8 in 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31"} : dispatch 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: pgmap v299: 228 pgs: 15 creating+peering, 17 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1024 KiB/s wr, 0 op/s 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-77452-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-77431-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: osdmap e236: 8 total, 8 up, 8 in 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31"} : dispatch 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:18.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:18 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:19.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:58:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:58:19.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31"}]': finished 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[41097]: osdmap e237: 8 total, 8 up, 8 in 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31"}]': finished 2024-09-21T11:58:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:19 smithi107 ceph-mon[30293]: osdmap e237: 8 total, 8 up, 8 in 2024-09-21T11:58:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:19 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-31"}]': finished 2024-09-21T11:58:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:19 smithi193 ceph-mon[33333]: osdmap e237: 8 total, 8 up, 8 in 2024-09-21T11:58:20.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: pgmap v302: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 514 B/s wr, 3 op/s 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[30293]: osdmap e238: 8 total, 8 up, 8 in 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: pgmap v302: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 514 B/s wr, 3 op/s 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:58:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:58:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:20.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:20 smithi107 ceph-mon[41097]: osdmap e238: 8 total, 8 up, 8 in 2024-09-21T11:58:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: pgmap v302: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 514 B/s wr, 3 op/s 2024-09-21T11:58:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:58:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:58:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:58:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:20.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-77452-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:20.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-77431-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:20.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:20 smithi193 ceph-mon[33333]: osdmap e238: 8 total, 8 up, 8 in 2024-09-21T11:58:22.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[30293]: osdmap e239: 8 total, 8 up, 8 in 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[30293]: pgmap v305: 244 pgs: 48 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[41097]: osdmap e239: 8 total, 8 up, 8 in 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[41097]: pgmap v305: 244 pgs: 48 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:22.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:22.755 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:58:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:58:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:22 smithi193 ceph-mon[33333]: osdmap e239: 8 total, 8 up, 8 in 2024-09-21T11:58:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-33","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:22 smithi193 ceph-mon[33333]: pgmap v305: 244 pgs: 48 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:22 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:23.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: osdmap e240: 8 total, 8 up, 8 in 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:23.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: osdmap e240: 8 total, 8 up, 8 in 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:23.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: osdmap e240: 8 total, 8 up, 8 in 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:23.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:23.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: osdmap e241: 8 total, 8 up, 8 in 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-33"} : dispatch 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[30293]: pgmap v308: 228 pgs: 13 unknown, 215 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: osdmap e241: 8 total, 8 up, 8 in 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-33"} : dispatch 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:24.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:24.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:24 smithi107 ceph-mon[41097]: pgmap v308: 228 pgs: 13 unknown, 215 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T11:58:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: osdmap e241: 8 total, 8 up, 8 in 2024-09-21T11:58:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-33"} : dispatch 2024-09-21T11:58:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/846775934' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"} : dispatch 2024-09-21T11:58:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3372770315' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"} : dispatch 2024-09-21T11:58:24.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:24 smithi193 ceph-mon[33333]: pgmap v308: 228 pgs: 13 unknown, 215 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-33"}]': finished 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: osdmap e242: 8 total, 8 up, 8 in 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-33", "mode": "writeback"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-33"}]': finished 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: osdmap e242: 8 total, 8 up, 8 in 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-33", "mode": "writeback"} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-33"}]': finished 2024-09-21T11:58:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-77431-40"}]': finished 2024-09-21T11:58:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-77452-49"}]': finished 2024-09-21T11:58:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: osdmap e242: 8 total, 8 up, 8 in 2024-09-21T11:58:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-33", "mode": "writeback"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:25.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-33", "mode": "writeback"}]': finished 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: osdmap e243: 8 total, 8 up, 8 in 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: pgmap v311: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-33", "mode": "writeback"}]': finished 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: osdmap e243: 8 total, 8 up, 8 in 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: pgmap v311: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:26.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:26.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-33", "mode": "writeback"}]': finished 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-77431-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-77452-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: osdmap e243: 8 total, 8 up, 8 in 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: pgmap v311: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:26.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: osdmap e244: 8 total, 8 up, 8 in 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33"} : dispatch 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33"}]': finished 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[30293]: osdmap e245: 8 total, 8 up, 8 in 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: osdmap e244: 8 total, 8 up, 8 in 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33"} : dispatch 2024-09-21T11:58:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33"}]': finished 2024-09-21T11:58:27.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:27 smithi107 ceph-mon[41097]: osdmap e245: 8 total, 8 up, 8 in 2024-09-21T11:58:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: osdmap e244: 8 total, 8 up, 8 in 2024-09-21T11:58:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33"} : dispatch 2024-09-21T11:58:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-77431-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:27.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-77452-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:27.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-33"}]': finished 2024-09-21T11:58:27.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:27 smithi193 ceph-mon[33333]: osdmap e245: 8 total, 8 up, 8 in 2024-09-21T11:58:28.669 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:28 smithi193 ceph-mon[33333]: pgmap v314: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:28.669 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:28 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:28.669 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:28 smithi193 ceph-mon[33333]: osdmap e246: 8 total, 8 up, 8 in 2024-09-21T11:58:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:28 smithi107 ceph-mon[30293]: pgmap v314: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:28 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:28 smithi107 ceph-mon[30293]: osdmap e246: 8 total, 8 up, 8 in 2024-09-21T11:58:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:28 smithi107 ceph-mon[41097]: pgmap v314: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:28 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:28 smithi107 ceph-mon[41097]: osdmap e246: 8 total, 8 up, 8 in 2024-09-21T11:58:29.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:58:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: osdmap e247: 8 total, 8 up, 8 in 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: osdmap e247: 8 total, 8 up, 8 in 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:29.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: osdmap e247: 8 total, 8 up, 8 in 2024-09-21T11:58:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-35","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:29.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:29.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: pgmap v317: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: osdmap e248: 8 total, 8 up, 8 in 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: pgmap v317: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: osdmap e248: 8 total, 8 up, 8 in 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: pgmap v317: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: osdmap e248: 8 total, 8 up, 8 in 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2078979425' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/689893892' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"} : dispatch 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"} : dispatch 2024-09-21T11:58:30.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:58:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: pgmap v319: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: osdmap e249: 8 total, 8 up, 8 in 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-77431-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: pgmap v319: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: osdmap e249: 8 total, 8 up, 8 in 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-77431-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: pgmap v319: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-77452-50"}]': finished 2024-09-21T11:58:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-77431-41"}]': finished 2024-09-21T11:58:32.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:32.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: osdmap e249: 8 total, 8 up, 8 in 2024-09-21T11:58:32.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:32.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:32.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:32.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-77431-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-77431-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: osdmap e250: 8 total, 8 up, 8 in 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-35", "mode": "writeback"} : dispatch 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi107-77431-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3890803092' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-35", "mode": "writeback"}]': finished 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-77431-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: osdmap e250: 8 total, 8 up, 8 in 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-35", "mode": "writeback"} : dispatch 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi107-77431-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3890803092' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:33.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-35", "mode": "writeback"}]': finished 2024-09-21T11:58:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-77431-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: osdmap e250: 8 total, 8 up, 8 in 2024-09-21T11:58:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-35", "mode": "writeback"} : dispatch 2024-09-21T11:58:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWrite_smithi107-77431-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3890803092' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:33.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:33.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:33.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-35", "mode": "writeback"}]': finished 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[41097]: pgmap v322: 260 pgs: 16 creating+peering, 34 unknown, 210 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[41097]: osdmap e251: 8 total, 8 up, 8 in 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-77431-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[41097]: osdmap e252: 8 total, 8 up, 8 in 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[30293]: pgmap v322: 260 pgs: 16 creating+peering, 34 unknown, 210 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[30293]: osdmap e251: 8 total, 8 up, 8 in 2024-09-21T11:58:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-77431-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:34.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:34 smithi107 ceph-mon[30293]: osdmap e252: 8 total, 8 up, 8 in 2024-09-21T11:58:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:34 smithi193 ceph-mon[33333]: pgmap v322: 260 pgs: 16 creating+peering, 34 unknown, 210 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-77452-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:34 smithi193 ceph-mon[33333]: osdmap e251: 8 total, 8 up, 8 in 2024-09-21T11:58:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:34 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-77431-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:34 smithi193 ceph-mon[33333]: osdmap e252: 8 total, 8 up, 8 in 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[30293]: osdmap e253: 8 total, 8 up, 8 in 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-77452-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[41097]: osdmap e253: 8 total, 8 up, 8 in 2024-09-21T11:58:35.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-77452-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:35 smithi193 ceph-mon[33333]: osdmap e253: 8 total, 8 up, 8 in 2024-09-21T11:58:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-77452-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[30293]: pgmap v325: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[30293]: osdmap e254: 8 total, 8 up, 8 in 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[41097]: pgmap v325: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:36.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:36.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[41097]: osdmap e254: 8 total, 8 up, 8 in 2024-09-21T11:58:36.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:36.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:36 smithi193 ceph-mon[33333]: pgmap v325: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:58:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:36 smithi193 ceph-mon[33333]: osdmap e254: 8 total, 8 up, 8 in 2024-09-21T11:58:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7046 ms) 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6969 ms) 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6988 ms) 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7049 ms) 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7029 ms) 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-21T11:58:38.386 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7013 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7017 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6902 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7051 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7025 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6930 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7035 ms) 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-21T11:58:38.387 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6966 ms) 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7039 ms) 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7024 ms) 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7018 ms) 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112101 ms total) 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] Global test environment tear-down 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (188746 ms total) 2024-09-21T11:58:38.388 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ PASSED ] 42 tests. 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: pgmap v328: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-77452-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: osdmap e255: 8 total, 8 up, 8 in 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:38.671 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: pgmap v328: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-77452-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: osdmap e255: 8 total, 8 up, 8 in 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: pgmap v328: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-77452-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: osdmap e255: 8 total, 8 up, 8 in 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"} : dispatch 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:38.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:39.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:58:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:58:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[41097]: osdmap e256: 8 total, 8 up, 8 in 2024-09-21T11:58:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:39.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:39.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[30293]: osdmap e256: 8 total, 8 up, 8 in 2024-09-21T11:58:39.656 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:39.656 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/855922603' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-77431-42"}]': finished 2024-09-21T11:58:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:39 smithi193 ceph-mon[33333]: osdmap e256: 8 total, 8 up, 8 in 2024-09-21T11:58:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:39.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[41097]: pgmap v331: 236 pgs: 4 creating+peering, 232 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[41097]: osdmap e257: 8 total, 8 up, 8 in 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[30293]: pgmap v331: 236 pgs: 4 creating+peering, 232 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T11:58:40.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:40.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:40.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[30293]: osdmap e257: 8 total, 8 up, 8 in 2024-09-21T11:58:40.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:40.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:40 smithi193 ceph-mon[33333]: pgmap v331: 236 pgs: 4 creating+peering, 232 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T11:58:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:40 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:40 smithi193 ceph-mon[33333]: osdmap e257: 8 total, 8 up, 8 in 2024-09-21T11:58:40.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35"} : dispatch 2024-09-21T11:58:40.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[30293]: osdmap e258: 8 total, 8 up, 8 in 2024-09-21T11:58:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[41097]: osdmap e258: 8 total, 8 up, 8 in 2024-09-21T11:58:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:41 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-35"}]': finished 2024-09-21T11:58:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:41 smithi193 ceph-mon[33333]: osdmap e258: 8 total, 8 up, 8 in 2024-09-21T11:58:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"} : dispatch 2024-09-21T11:58:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:58:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: pgmap v334: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: osdmap e259: 8 total, 8 up, 8 in 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: pgmap v334: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: osdmap e259: 8 total, 8 up, 8 in 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:42.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:42 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: pgmap v334: 228 pgs: 228 active+clean; 4.6 MiB data, 571 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T11:58:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3552847958' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-77452-52"}]': finished 2024-09-21T11:58:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: osdmap e259: 8 total, 8 up, 8 in 2024-09-21T11:58:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:42 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[41097]: osdmap e260: 8 total, 8 up, 8 in 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[30293]: osdmap e260: 8 total, 8 up, 8 in 2024-09-21T11:58:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:43.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:43.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:43 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:43 smithi193 ceph-mon[33333]: osdmap e260: 8 total, 8 up, 8 in 2024-09-21T11:58:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-37","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:43 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[30293]: pgmap v337: 228 pgs: 17 creating+peering, 15 unknown, 196 active+clean; 4.6 MiB data, 572 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[30293]: osdmap e261: 8 total, 8 up, 8 in 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[41097]: pgmap v337: 228 pgs: 17 creating+peering, 15 unknown, 196 active+clean; 4.6 MiB data, 572 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[41097]: osdmap e261: 8 total, 8 up, 8 in 2024-09-21T11:58:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:44 smithi193 ceph-mon[33333]: pgmap v337: 228 pgs: 17 creating+peering, 15 unknown, 196 active+clean; 4.6 MiB data, 572 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:58:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:44 smithi193 ceph-mon[33333]: osdmap e261: 8 total, 8 up, 8 in 2024-09-21T11:58:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[30293]: osdmap e262: 8 total, 8 up, 8 in 2024-09-21T11:58:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-37"} : dispatch 2024-09-21T11:58:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[41097]: osdmap e262: 8 total, 8 up, 8 in 2024-09-21T11:58:45.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-37"} : dispatch 2024-09-21T11:58:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:45 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-77452-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:45 smithi193 ceph-mon[33333]: osdmap e262: 8 total, 8 up, 8 in 2024-09-21T11:58:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-37"} : dispatch 2024-09-21T11:58:46.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[30293]: pgmap v340: 236 pgs: 8 unknown, 20 creating+peering, 208 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-37"}]': finished 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[30293]: osdmap e263: 8 total, 8 up, 8 in 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-37", "mode": "writeback"} : dispatch 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[41097]: pgmap v340: 236 pgs: 8 unknown, 20 creating+peering, 208 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-37"}]': finished 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[41097]: osdmap e263: 8 total, 8 up, 8 in 2024-09-21T11:58:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-37", "mode": "writeback"} : dispatch 2024-09-21T11:58:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:46 smithi193 ceph-mon[33333]: pgmap v340: 236 pgs: 8 unknown, 20 creating+peering, 208 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:58:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-37"}]': finished 2024-09-21T11:58:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:46 smithi193 ceph-mon[33333]: osdmap e263: 8 total, 8 up, 8 in 2024-09-21T11:58:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-37", "mode": "writeback"} : dispatch 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-37", "mode": "writeback"}]': finished 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: osdmap e264: 8 total, 8 up, 8 in 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: osdmap e265: 8 total, 8 up, 8 in 2024-09-21T11:58:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37"} : dispatch 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-37", "mode": "writeback"}]': finished 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: osdmap e264: 8 total, 8 up, 8 in 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:47.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:47.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:47.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: osdmap e265: 8 total, 8 up, 8 in 2024-09-21T11:58:47.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37"} : dispatch 2024-09-21T11:58:47.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:47 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-37", "mode": "writeback"}]': finished 2024-09-21T11:58:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: osdmap e264: 8 total, 8 up, 8 in 2024-09-21T11:58:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: osdmap e265: 8 total, 8 up, 8 in 2024-09-21T11:58:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1807119206' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37"} : dispatch 2024-09-21T11:58:47.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:47 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"} : dispatch 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: pgmap v344: 228 pgs: 20 creating+peering, 208 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37"}]': finished 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: osdmap e266: 8 total, 8 up, 8 in 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: pgmap v344: 228 pgs: 20 creating+peering, 208 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37"}]': finished 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: osdmap e266: 8 total, 8 up, 8 in 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:48.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:48.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:58:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: pgmap v344: 228 pgs: 20 creating+peering, 208 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-37"}]': finished 2024-09-21T11:58:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-77452-53"}]': finished 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: osdmap e266: 8 total, 8 up, 8 in 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:48.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[30293]: osdmap e267: 8 total, 8 up, 8 in 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[41097]: osdmap e267: 8 total, 8 up, 8 in 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-77452-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:49 smithi193 ceph-mon[33333]: osdmap e267: 8 total, 8 up, 8 in 2024-09-21T11:58:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:50.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:50 smithi107 ceph-mon[41097]: pgmap v347: 196 pgs: 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-21T11:58:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:50 smithi107 ceph-mon[41097]: osdmap e268: 8 total, 8 up, 8 in 2024-09-21T11:58:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:50 smithi107 ceph-mon[30293]: pgmap v347: 196 pgs: 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-21T11:58:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:50 smithi107 ceph-mon[30293]: osdmap e268: 8 total, 8 up, 8 in 2024-09-21T11:58:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:50 smithi193 ceph-mon[33333]: pgmap v347: 196 pgs: 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-21T11:58:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:50 smithi193 ceph-mon[33333]: osdmap e268: 8 total, 8 up, 8 in 2024-09-21T11:58:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-39","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:58:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[41097]: osdmap e269: 8 total, 8 up, 8 in 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[41097]: pgmap v350: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[30293]: osdmap e269: 8 total, 8 up, 8 in 2024-09-21T11:58:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:52 smithi107 ceph-mon[30293]: pgmap v350: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T11:58:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-77452-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:52 smithi193 ceph-mon[33333]: osdmap e269: 8 total, 8 up, 8 in 2024-09-21T11:58:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:52 smithi193 ceph-mon[33333]: pgmap v350: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T11:58:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[30293]: osdmap e270: 8 total, 8 up, 8 in 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-39"} : dispatch 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[41097]: osdmap e270: 8 total, 8 up, 8 in 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-39"} : dispatch 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:58:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:53 smithi193 ceph-mon[33333]: osdmap e270: 8 total, 8 up, 8 in 2024-09-21T11:58:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-39"} : dispatch 2024-09-21T11:58:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:53 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:58:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:54.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-39"}]': finished 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: osdmap e271: 8 total, 8 up, 8 in 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-39", "mode": "writeback"} : dispatch 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: pgmap v353: 228 pgs: 16 unknown, 212 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-39", "mode": "writeback"}]': finished 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[41097]: osdmap e272: 8 total, 8 up, 8 in 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-39"}]': finished 2024-09-21T11:58:54.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: osdmap e271: 8 total, 8 up, 8 in 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-39", "mode": "writeback"} : dispatch 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: pgmap v353: 228 pgs: 16 unknown, 212 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-39", "mode": "writeback"}]': finished 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:54.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:54 smithi107 ceph-mon[30293]: osdmap e272: 8 total, 8 up, 8 in 2024-09-21T11:58:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-39"}]': finished 2024-09-21T11:58:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: osdmap e271: 8 total, 8 up, 8 in 2024-09-21T11:58:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-39", "mode": "writeback"} : dispatch 2024-09-21T11:58:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: pgmap v353: 228 pgs: 16 unknown, 212 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:58:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-39", "mode": "writeback"}]': finished 2024-09-21T11:58:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:54.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:54 smithi193 ceph-mon[33333]: osdmap e272: 8 total, 8 up, 8 in 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:55.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:55.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:55.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:55.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:55.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:55 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:58:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3563184097' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"} : dispatch 2024-09-21T11:58:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:58:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-77452-54"}]': finished 2024-09-21T11:58:55.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:58:56.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: osdmap e273: 8 total, 8 up, 8 in 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39"} : dispatch 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-77452-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39"}]': finished 2024-09-21T11:58:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-77452-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: osdmap e273: 8 total, 8 up, 8 in 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39"} : dispatch 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-77452-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39"}]': finished 2024-09-21T11:58:56.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-77452-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: osdmap e273: 8 total, 8 up, 8 in 2024-09-21T11:58:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39"} : dispatch 2024-09-21T11:58:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:58:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:56.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:56.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-77452-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:58:56.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:58:56.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-39"}]': finished 2024-09-21T11:58:56.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-77452-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:58:57.357 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: seed 77755 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (321 ms) 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10144 ms) 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (53 ms) 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (10518 ms total) 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-21T11:58:57.358 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (6972 ms) 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8014 ms) 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10006 ms) 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [3] 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [4,3] 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [5,4,3] 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting some heads 2024-09-21T11:58:57.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 3 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: waiting for scrubs... 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: done waiting 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46049 ms) 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9974 ms) 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8030 ms) 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-21T11:58:57.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8027 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8027 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10016 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8942 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10028 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12989 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7952 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8038 ms) 2024-09-21T11:58:57.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-21T11:58:57.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13059 ms) 2024-09-21T11:58:57.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-21T11:58:57.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7905 ms) 2024-09-21T11:58:57.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-21T11:58:57.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8044 ms) 2024-09-21T11:58:57.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:57 smithi107 ceph-mon[41097]: osdmap e274: 8 total, 8 up, 8 in 2024-09-21T11:58:57.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi107-77452-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:57 smithi107 ceph-mon[30293]: osdmap e274: 8 total, 8 up, 8 in 2024-09-21T11:58:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi107-77452-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:57 smithi193 ceph-mon[33333]: osdmap e274: 8 total, 8 up, 8 in 2024-09-21T11:58:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "IsSafePP_smithi107-77452-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:58:58.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:58 smithi193 ceph-mon[33333]: osdmap e275: 8 total, 8 up, 8 in 2024-09-21T11:58:58.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:58 smithi193 ceph-mon[33333]: pgmap v359: 196 pgs: 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:58.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-77452-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:58:58.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:58 smithi193 ceph-mon[33333]: osdmap e276: 8 total, 8 up, 8 in 2024-09-21T11:58:58.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:58.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[30293]: osdmap e275: 8 total, 8 up, 8 in 2024-09-21T11:58:58.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[30293]: pgmap v359: 196 pgs: 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-77452-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[30293]: osdmap e276: 8 total, 8 up, 8 in 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[41097]: osdmap e275: 8 total, 8 up, 8 in 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[41097]: pgmap v359: 196 pgs: 196 active+clean; 4.6 MiB data, 577 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-77452-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:58:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[41097]: osdmap e276: 8 total, 8 up, 8 in 2024-09-21T11:58:58.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-41","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:58:59.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:58:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:58:59.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[30293]: osdmap e277: 8 total, 8 up, 8 in 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[41097]: osdmap e277: 8 total, 8 up, 8 in 2024-09-21T11:58:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:58:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:58:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:58:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:59 smithi193 ceph-mon[33333]: osdmap e277: 8 total, 8 up, 8 in 2024-09-21T11:58:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:58:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[30293]: pgmap v362: 236 pgs: 33 creating+peering, 7 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[30293]: osdmap e278: 8 total, 8 up, 8 in 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[41097]: pgmap v362: 236 pgs: 33 creating+peering, 7 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[41097]: osdmap e278: 8 total, 8 up, 8 in 2024-09-21T11:59:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T11:59:00.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:59:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:00 smithi193 ceph-mon[33333]: pgmap v362: 236 pgs: 33 creating+peering, 7 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-21T11:59:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:00 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:00 smithi193 ceph-mon[33333]: osdmap e278: 8 total, 8 up, 8 in 2024-09-21T11:59:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T11:59:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:59:02.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:59:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_count","val": "2"}]': finished 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[30293]: osdmap e279: 8 total, 8 up, 8 in 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[30293]: pgmap v365: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_count","val": "2"}]': finished 2024-09-21T11:59:02.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:59:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[41097]: osdmap e279: 8 total, 8 up, 8 in 2024-09-21T11:59:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:59:02.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:02 smithi107 ceph-mon[41097]: pgmap v365: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-21T11:59:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_count","val": "2"}]': finished 2024-09-21T11:59:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:59:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:02 smithi193 ceph-mon[33333]: osdmap e279: 8 total, 8 up, 8 in 2024-09-21T11:59:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"} : dispatch 2024-09-21T11:59:02.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:02 smithi193 ceph-mon[33333]: pgmap v365: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: osdmap e280: 8 total, 8 up, 8 in 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:03.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: osdmap e280: 8 total, 8 up, 8 in 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:03.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/179523303' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-77452-55"}]': finished 2024-09-21T11:59:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: osdmap e280: 8 total, 8 up, 8 in 2024-09-21T11:59:03.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T11:59:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:03.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:04.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[41097]: pgmap v367: 228 pgs: 19 creating+peering, 209 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[41097]: osdmap e281: 8 total, 8 up, 8 in 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[30293]: pgmap v367: 228 pgs: 19 creating+peering, 209 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T11:59:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:04.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[30293]: osdmap e281: 8 total, 8 up, 8 in 2024-09-21T11:59:04.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:04.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:04 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:04 smithi193 ceph-mon[33333]: pgmap v367: 228 pgs: 19 creating+peering, 209 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T11:59:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-77452-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:04 smithi193 ceph-mon[33333]: osdmap e281: 8 total, 8 up, 8 in 2024-09-21T11:59:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:04.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:04 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:05.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[30293]: osdmap e282: 8 total, 8 up, 8 in 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41"} : dispatch 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41"}]': finished 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[30293]: osdmap e283: 8 total, 8 up, 8 in 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[41097]: osdmap e282: 8 total, 8 up, 8 in 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41"} : dispatch 2024-09-21T11:59:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41"}]': finished 2024-09-21T11:59:05.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:05 smithi107 ceph-mon[41097]: osdmap e283: 8 total, 8 up, 8 in 2024-09-21T11:59:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:05 smithi193 ceph-mon[33333]: osdmap e282: 8 total, 8 up, 8 in 2024-09-21T11:59:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41"} : dispatch 2024-09-21T11:59:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-77452-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-41"}]': finished 2024-09-21T11:59:05.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:05 smithi193 ceph-mon[33333]: osdmap e283: 8 total, 8 up, 8 in 2024-09-21T11:59:06.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:06 smithi107 ceph-mon[30293]: pgmap v370: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:06 smithi107 ceph-mon[30293]: osdmap e284: 8 total, 8 up, 8 in 2024-09-21T11:59:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:06 smithi107 ceph-mon[41097]: pgmap v370: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:06 smithi107 ceph-mon[41097]: osdmap e284: 8 total, 8 up, 8 in 2024-09-21T11:59:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:06 smithi193 ceph-mon[33333]: pgmap v370: 228 pgs: 228 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:06 smithi193 ceph-mon[33333]: osdmap e284: 8 total, 8 up, 8 in 2024-09-21T11:59:07.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[30293]: osdmap e285: 8 total, 8 up, 8 in 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[41097]: osdmap e285: 8 total, 8 up, 8 in 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:07 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:07 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:07 smithi193 ceph-mon[33333]: osdmap e285: 8 total, 8 up, 8 in 2024-09-21T11:59:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-43","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:07 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.703 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: pgmap v373: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: osdmap e286: 8 total, 8 up, 8 in 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi107-77755-6","var": "pg_num","format": "json"} : dispatch 2024-09-21T11:59:08.704 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: pgmap v373: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: osdmap e286: 8 total, 8 up, 8 in 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi107-77755-6","var": "pg_num","format": "json"} : dispatch 2024-09-21T11:59:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: pgmap v373: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: osdmap e286: 8 total, 8 up, 8 in 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2229208513' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"} : dispatch 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi107-77755-6","var": "pg_num","format": "json"} : dispatch 2024-09-21T11:59:08.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:09.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:59:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:59:09.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[41097]: osdmap e287: 8 total, 8 up, 8 in 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[30293]: osdmap e287: 8 total, 8 up, 8 in 2024-09-21T11:59:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-21T11:59:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-77452-56"}]': finished 2024-09-21T11:59:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:09 smithi193 ceph-mon[33333]: osdmap e287: 8 total, 8 up, 8 in 2024-09-21T11:59:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_count","val": "8"} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: pgmap v376: 228 pgs: 15 creating+peering, 213 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_count","val": "8"}]': finished 2024-09-21T11:59:10.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: osdmap e288: 8 total, 8 up, 8 in 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: pgmap v376: 228 pgs: 15 creating+peering, 213 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:10.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_count","val": "8"}]': finished 2024-09-21T11:59:10.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:10.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: osdmap e288: 8 total, 8 up, 8 in 2024-09-21T11:59:10.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:10.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.756 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:10 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: pgmap v376: 228 pgs: 15 creating+peering, 213 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_count","val": "8"}]': finished 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-77452-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: osdmap e288: 8 total, 8 up, 8 in 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:10.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:10 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: ok, hit_set contains 274:602f83fe:::foo:head 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9027 ms) 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg_num = 32 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 0 ls 1726919953,0 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 1 ls 1726919953,0 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 2 ls 1726919953,0 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 3 ls 1726919953,0 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 4 ls 1726919953,0 2024-09-21T11:59:12.619 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 5 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 6 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 7 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 8 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 9 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 10 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 11 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 12 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 13 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 14 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 15 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 16 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 17 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 18 ls 1726919953,0 2024-09-21T11:59:12.620 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 19 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 20 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 21 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 22 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 23 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 24 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 25 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 26 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 27 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 28 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 29 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 30 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 31 ls 1726919953,0 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg_num = 32 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6cac518f:::0:head 2024-09-21T11:59:12.621 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:02547ec2:::1:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f905c69b:::2:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:cfc208b3:::3:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d83876eb:::4:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b29083e3:::5:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c4fdafeb:::6:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:5c6b0b28:::7:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:bd63b0f1:::8:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e960b815:::9:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:52ea6a34:::10:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:89d3ae78:::11:head 2024-09-21T11:59:12.622 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:de5d7c5f:::12:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:566253c9:::13:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:62a1935d:::14:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:863748b0:::15:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3958e169:::16:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4d4dabf9:::17:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:8391935d:::18:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:28883081:::19:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:69259c59:::20:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4bdb80b7:::21:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a11c5d71:::22:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:271af37b:::23:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:95b121be:::24:head 2024-09-21T11:59:12.623 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:58d1031b:::25:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:0a050783:::26:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c709704c:::27:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:cbe56eaf:::28:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:86b4b162:::29:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:70d89383:::30:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:dd450c7c:::31:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6d5729b1:::32:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c388f3fb:::33:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:56cfea31:::34:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9dbc1bf7:::35:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:40b74ccd:::36:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4d5aaf42:::37:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:920f362c:::38:head 2024-09-21T11:59:12.624 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6cc53222:::39:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9cad833f:::40:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:1ea84d41:::41:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c4480ef6:::42:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a694361e:::43:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d1bd33e9:::44:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ddc2cd5d:::45:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:2b782207:::46:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7b187fca:::47:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:90ecdf6f:::48:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a5ed95fe:::49:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ea0eaa55:::50:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f33ef17b:::51:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a0d1b2f6:::52:head 2024-09-21T11:59:12.625 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:60c5229e:::53:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:edcbc575:::54:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:102cf253:::55:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:efb7fb0b:::56:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:50d0a326:::57:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d4dc5daf:::58:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3a130462:::59:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ec87ed71:::60:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d5bc9454:::61:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3ddfe313:::62:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7c2816b9:::63:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:47e00e4d:::64:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c6410c18:::65:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b48ed237:::66:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:cd63ad31:::67:head 2024-09-21T11:59:12.626 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b179e92b:::68:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:0d9f741a:::69:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6d3352ae:::70:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c6d5c19e:::71:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:bc4729c3:::72:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:77e930b9:::73:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:0abeecfd:::74:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b7c37e15:::75:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b6378398:::76:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:02bd68de:::77:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:cc795d2d:::78:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:630d4fea:::79:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e0d29ef5:::80:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:fd6f13d2:::81:head 2024-09-21T11:59:12.627 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:606461d5:::82:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:eadbdc43:::83:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:8761d0bb:::84:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9ef0186f:::85:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e0d41294:::86:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:961de695:::87:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:1423148f:::88:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:633a8fa2:::89:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a8653809:::90:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3dac8b33:::91:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:35aad435:::92:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f6dcc343:::93:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:dbbdad87:::94:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:1cb48ce0:::95:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:03cd461c:::96:head 2024-09-21T11:59:12.628 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:17a4ea99:::97:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9993c9a7:::98:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6394211c:::99:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:94c7ae57:::100:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6fdee5bb:::101:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9a477fd1:::102:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:eb850916:::103:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:affc56b9:::104:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b42dc814:::105:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f319f8f0:::106:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9a40b9de:::107:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:8b524f28:::108:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e3de589f:::109:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:90f90a5b:::110:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a7b4f1d7:::111:head 2024-09-21T11:59:12.629 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:af51766e:::112:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b6f90bd1:::113:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e0261208:::114:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c9569ef7:::115:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:61bebe50:::116:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:fe93412b:::117:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d3d38bee:::118:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3100ba0c:::119:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d0560ada:::120:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f0ea8b35:::121:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:766f231a:::122:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a07a2582:::123:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:bd7c6b3a:::124:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:fb2ddaff:::125:head 2024-09-21T11:59:12.630 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4408e1fe:::126:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ee1df7a7:::127:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c3002909:::128:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4f48ffa9:::129:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:edf38733:::130:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c08425c0:::131:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:5f902d98:::132:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:41ea2c93:::133:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:813cee13:::134:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:0131818d:::135:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:26ba5a85:::136:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:381b8a5a:::137:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:28797e47:::138:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:bfca7f22:::139:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:36807075:::140:head 2024-09-21T11:59:12.631 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:80b03975:::141:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:5c15709b:::142:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f39ea15e:::143:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ea992956:::144:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:48887b1c:::145:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:9f24a9dd:::146:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:987f100b:::147:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d2dd3581:::148:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7fed1808:::149:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c80b70e9:::150:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:85ed90f9:::151:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:36428b24:::152:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d044c34a:::153:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7c18bf58:::154:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d1c21232:::155:head 2024-09-21T11:59:12.632 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a7a3c575:::156:head 2024-09-21T11:59:12.633 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:87da0633:::157:head 2024-09-21T11:59:12.633 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d5ac3822:::158:head 2024-09-21T11:59:12.633 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3f20522d:::159:head 2024-09-21T11:59:12.633 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6ca26563:::160:head 2024-09-21T11:59:12.633 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:532ce135:::161:head 2024-09-21T11:59:12.633 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:c78863e6:::162:head 2024-09-21T11:59:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:59:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[30293]: pgmap v379: 228 pgs: 15 creating+peering, 213 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[30293]: osdmap e289: 8 total, 8 up, 8 in 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[41097]: pgmap v379: 228 pgs: 15 creating+peering, 213 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:12.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[41097]: osdmap e289: 8 total, 8 up, 8 in 2024-09-21T11:59:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-21T11:59:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:12 smithi193 ceph-mon[33333]: pgmap v379: 228 pgs: 15 creating+peering, 213 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:12 smithi193 ceph-mon[33333]: osdmap e289: 8 total, 8 up, 8 in 2024-09-21T11:59:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_type","val": "explicit_hash"} : dispatch 2024-09-21T11:59:13.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:13.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-21T11:59:13.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[30293]: osdmap e290: 8 total, 8 up, 8 in 2024-09-21T11:59:13.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi107-77755-43","var": "pg_num","format": "json"} : dispatch 2024-09-21T11:59:13.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:13.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43"} : dispatch 2024-09-21T11:59:13.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:13.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-21T11:59:13.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[41097]: osdmap e290: 8 total, 8 up, 8 in 2024-09-21T11:59:13.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi107-77755-43","var": "pg_num","format": "json"} : dispatch 2024-09-21T11:59:13.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:13.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43"} : dispatch 2024-09-21T11:59:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:13 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-77452-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-21T11:59:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:13 smithi193 ceph-mon[33333]: osdmap e290: 8 total, 8 up, 8 in 2024-09-21T11:59:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool get","pool":"test-rados-api-smithi107-77755-43","var": "pg_num","format": "json"} : dispatch 2024-09-21T11:59:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:13.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43"} : dispatch 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[30293]: pgmap v382: 236 pgs: 3 creating+peering, 5 unknown, 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43"}]': finished 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[30293]: osdmap e291: 8 total, 8 up, 8 in 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[41097]: pgmap v382: 236 pgs: 3 creating+peering, 5 unknown, 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43"}]': finished 2024-09-21T11:59:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:14 smithi107 ceph-mon[41097]: osdmap e291: 8 total, 8 up, 8 in 2024-09-21T11:59:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:14 smithi193 ceph-mon[33333]: pgmap v382: 236 pgs: 3 creating+peering, 5 unknown, 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-21T11:59:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:14 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-43"}]': finished 2024-09-21T11:59:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:14 smithi193 ceph-mon[33333]: osdmap e291: 8 total, 8 up, 8 in 2024-09-21T11:59:15.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:15 smithi107 ceph-mon[30293]: osdmap e292: 8 total, 8 up, 8 in 2024-09-21T11:59:15.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:15.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:15 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:15 smithi107 ceph-mon[41097]: osdmap e292: 8 total, 8 up, 8 in 2024-09-21T11:59:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:15.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:15 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:15.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:15 smithi193 ceph-mon[33333]: osdmap e292: 8 total, 8 up, 8 in 2024-09-21T11:59:15.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:15.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:15 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[30293]: pgmap v385: 196 pgs: 196 active+clean; 8.4 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[30293]: osdmap e293: 8 total, 8 up, 8 in 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:16.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[41097]: pgmap v385: 196 pgs: 196 active+clean; 8.4 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[41097]: osdmap e293: 8 total, 8 up, 8 in 2024-09-21T11:59:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:16.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:16 smithi193 ceph-mon[33333]: pgmap v385: 196 pgs: 196 active+clean; 8.4 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:16 smithi193 ceph-mon[33333]: osdmap e293: 8 total, 8 up, 8 in 2024-09-21T11:59:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-45","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3499148322' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"} : dispatch 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: osdmap e294: 8 total, 8 up, 8 in 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: osdmap e295: 8 total, 8 up, 8 in 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:17.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: osdmap e294: 8 total, 8 up, 8 in 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: osdmap e295: 8 total, 8 up, 8 in 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-77452-57"}]': finished 2024-09-21T11:59:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: osdmap e294: 8 total, 8 up, 8 in 2024-09-21T11:59:17.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-77452-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: osdmap e295: 8 total, 8 up, 8 in 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_count","val": "3"} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:17.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:18.711 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:18 smithi193 ceph-mon[33333]: pgmap v388: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:18.711 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_count","val": "3"}]': finished 2024-09-21T11:59:18.711 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:18 smithi193 ceph-mon[33333]: osdmap e296: 8 total, 8 up, 8 in 2024-09-21T11:59:18.711 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[41097]: pgmap v388: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_count","val": "3"}]': finished 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[41097]: osdmap e296: 8 total, 8 up, 8 in 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[30293]: pgmap v388: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_count","val": "3"}]': finished 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[30293]: osdmap e296: 8 total, 8 up, 8 in 2024-09-21T11:59:18.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_period","val": "3"} : dispatch 2024-09-21T11:59:19.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:59:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:59:19.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_period","val": "3"}]': finished 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[41097]: osdmap e297: 8 total, 8 up, 8 in 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_period","val": "3"}]': finished 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[30293]: osdmap e297: 8 total, 8 up, 8 in 2024-09-21T11:59:19.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T11:59:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:19 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-77452-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_period","val": "3"}]': finished 2024-09-21T11:59:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:19 smithi193 ceph-mon[33333]: osdmap e297: 8 total, 8 up, 8 in 2024-09-21T11:59:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[41097]: pgmap v391: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[41097]: osdmap e298: 8 total, 8 up, 8 in 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[30293]: pgmap v391: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T11:59:20.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[30293]: osdmap e298: 8 total, 8 up, 8 in 2024-09-21T11:59:20.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-21T11:59:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:20 smithi193 ceph-mon[33333]: pgmap v391: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:20 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:59:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T11:59:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:20 smithi193 ceph-mon[33333]: osdmap e298: 8 total, 8 up, 8 in 2024-09-21T11:59:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-21T11:59:22.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: osdmap e299: 8 total, 8 up, 8 in 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: osdmap e299: 8 total, 8 up, 8 in 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:22.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:21 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:59:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:59:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T11:59:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-21T11:59:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: osdmap e299: 8 total, 8 up, 8 in 2024-09-21T11:59:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:22.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:21 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:22.659 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:59:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[30293]: pgmap v394: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[30293]: osdmap e300: 8 total, 8 up, 8 in 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[41097]: pgmap v394: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:23.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:23.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:23.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[41097]: osdmap e300: 8 total, 8 up, 8 in 2024-09-21T11:59:23.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:23.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:22 smithi193 ceph-mon[33333]: pgmap v394: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:22 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:22 smithi193 ceph-mon[33333]: osdmap e300: 8 total, 8 up, 8 in 2024-09-21T11:59:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3990430937' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:23.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: osdmap e301: 8 total, 8 up, 8 in 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: osdmap e301: 8 total, 8 up, 8 in 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:24.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-77452-58"}]': finished 2024-09-21T11:59:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: osdmap e301: 8 total, 8 up, 8 in 2024-09-21T11:59:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:24.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:24.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[30293]: pgmap v397: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[30293]: osdmap e302: 8 total, 8 up, 8 in 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[41097]: pgmap v397: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[41097]: osdmap e302: 8 total, 8 up, 8 in 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:24 smithi193 ceph-mon[33333]: pgmap v397: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:24 smithi193 ceph-mon[33333]: osdmap e302: 8 total, 8 up, 8 in 2024-09-21T11:59:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:26.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:26 smithi107 ceph-mon[30293]: pgmap v400: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-21T11:59:26.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:26 smithi107 ceph-mon[30293]: osdmap e303: 8 total, 8 up, 8 in 2024-09-21T11:59:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:26 smithi107 ceph-mon[41097]: pgmap v400: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-21T11:59:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:26 smithi107 ceph-mon[41097]: osdmap e303: 8 total, 8 up, 8 in 2024-09-21T11:59:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:26 smithi193 ceph-mon[33333]: pgmap v400: 228 pgs: 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-21T11:59:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:26 smithi193 ceph-mon[33333]: osdmap e303: 8 total, 8 up, 8 in 2024-09-21T11:59:27.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:27 smithi107 ceph-mon[30293]: osdmap e304: 8 total, 8 up, 8 in 2024-09-21T11:59:27.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:27 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:27 smithi107 ceph-mon[41097]: osdmap e304: 8 total, 8 up, 8 in 2024-09-21T11:59:27.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:27 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-77452-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:27 smithi193 ceph-mon[33333]: osdmap e304: 8 total, 8 up, 8 in 2024-09-21T11:59:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:27 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:28.722 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:28 smithi193 ceph-mon[33333]: pgmap v403: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T11:59:28.722 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:28 smithi193 ceph-mon[33333]: osdmap e305: 8 total, 8 up, 8 in 2024-09-21T11:59:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:28 smithi107 ceph-mon[30293]: pgmap v403: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T11:59:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:28 smithi107 ceph-mon[30293]: osdmap e305: 8 total, 8 up, 8 in 2024-09-21T11:59:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:28 smithi107 ceph-mon[41097]: pgmap v403: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T11:59:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:28 smithi107 ceph-mon[41097]: osdmap e305: 8 total, 8 up, 8 in 2024-09-21T11:59:29.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:59:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[41097]: osdmap e306: 8 total, 8 up, 8 in 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[30293]: osdmap e306: 8 total, 8 up, 8 in 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:29 smithi193 ceph-mon[33333]: osdmap e306: 8 total, 8 up, 8 in 2024-09-21T11:59:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:29.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7462ddf6:::.RoundTripAppendPP (3001 ms) 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3021 ms) 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3011 ms) 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3031 ms) 2024-09-21T11:59:30.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: setting pool EIO 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: max_success 100, min_failed 101 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4018 ms) 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3009 ms) 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (113731 ms total) 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2999 ms) 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9015 ms) 2024-09-21T11:59:30.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14965 ms) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3003 ms) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29984 ms total) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3008 ms) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3008 ms total) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13969 ms) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6966 ms) 2024-09-21T11:59:30.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7040 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7023 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3009 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7038 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6904 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7034 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7025 ms) 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-21T11:59:30.485 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7019 ms) 2024-09-21T11:59:30.486 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-21T11:59:30.486 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7038 ms) 2024-09-21T11:59:30.486 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-21T11:59:30.486 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7012 ms) 2024-09-21T11:59:30.486 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-21T11:59:30.486 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7039 ms) 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[30293]: pgmap v406: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[30293]: osdmap e307: 8 total, 8 up, 8 in 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[41097]: pgmap v406: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[41097]: osdmap e307: 8 total, 8 up, 8 in 2024-09-21T11:59:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:30.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:30 smithi193 ceph-mon[33333]: pgmap v406: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:30 smithi193 ceph-mon[33333]: osdmap e307: 8 total, 8 up, 8 in 2024-09-21T11:59:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2877811748' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"} : dispatch 2024-09-21T11:59:31.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:31.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:31 smithi107 ceph-mon[41097]: osdmap e308: 8 total, 8 up, 8 in 2024-09-21T11:59:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:31 smithi107 ceph-mon[30293]: osdmap e308: 8 total, 8 up, 8 in 2024-09-21T11:59:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-77452-59"}]': finished 2024-09-21T11:59:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:31 smithi193 ceph-mon[33333]: osdmap e308: 8 total, 8 up, 8 in 2024-09-21T11:59:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:59:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[30293]: pgmap v409: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[30293]: osdmap e309: 8 total, 8 up, 8 in 2024-09-21T11:59:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3142008232' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[41097]: pgmap v409: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[41097]: osdmap e309: 8 total, 8 up, 8 in 2024-09-21T11:59:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3142008232' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:32 smithi193 ceph-mon[33333]: pgmap v409: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:32 smithi193 ceph-mon[33333]: osdmap e309: 8 total, 8 up, 8 in 2024-09-21T11:59:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3142008232' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:32.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:33.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:33 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:33 smithi107 ceph-mon[30293]: osdmap e310: 8 total, 8 up, 8 in 2024-09-21T11:59:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:33 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:33 smithi107 ceph-mon[41097]: osdmap e310: 8 total, 8 up, 8 in 2024-09-21T11:59:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:33 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-77452-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:33 smithi193 ceph-mon[33333]: osdmap e310: 8 total, 8 up, 8 in 2024-09-21T11:59:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: pgmap v412: 260 pgs: 18 unknown, 242 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:59:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: osdmap e311: 8 total, 8 up, 8 in 2024-09-21T11:59:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45"} : dispatch 2024-09-21T11:59:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:34.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: pgmap v412: 260 pgs: 18 unknown, 242 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: osdmap e311: 8 total, 8 up, 8 in 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45"} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: pgmap v412: 260 pgs: 18 unknown, 242 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: osdmap e311: 8 total, 8 up, 8 in 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45"} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:35.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45"}]': finished 2024-09-21T11:59:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:35 smithi193 ceph-mon[33333]: osdmap e312: 8 total, 8 up, 8 in 2024-09-21T11:59:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:36.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45"}]': finished 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[30293]: osdmap e312: 8 total, 8 up, 8 in 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-45"}]': finished 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-77452-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[41097]: osdmap e312: 8 total, 8 up, 8 in 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:36 smithi193 ceph-mon[33333]: pgmap v415: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:36 smithi193 ceph-mon[33333]: osdmap e313: 8 total, 8 up, 8 in 2024-09-21T11:59:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:36 smithi193 ceph-mon[33333]: osdmap e314: 8 total, 8 up, 8 in 2024-09-21T11:59:36.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[41097]: pgmap v415: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[41097]: osdmap e313: 8 total, 8 up, 8 in 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[41097]: osdmap e314: 8 total, 8 up, 8 in 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[30293]: pgmap v415: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[30293]: osdmap e313: 8 total, 8 up, 8 in 2024-09-21T11:59:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-77452-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:37.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[30293]: osdmap e314: 8 total, 8 up, 8 in 2024-09-21T11:59:37.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-47","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:38.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:59:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-mon[33333]: pgmap v418: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-mon[33333]: osdmap e315: 8 total, 8 up, 8 in 2024-09-21T11:59:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[41097]: pgmap v418: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[41097]: osdmap e315: 8 total, 8 up, 8 in 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[30293]: pgmap v418: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[30293]: osdmap e315: 8 total, 8 up, 8 in 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:39.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:39 smithi193 ceph-mon[33333]: osdmap e316: 8 total, 8 up, 8 in 2024-09-21T11:59:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:39.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:39 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:39.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:39.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:39.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[41097]: osdmap e316: 8 total, 8 up, 8 in 2024-09-21T11:59:39.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:39.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[30293]: osdmap e316: 8 total, 8 up, 8 in 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:39.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-21T11:59:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-47"}]': finished 2024-09-21T11:59:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: osdmap e317: 8 total, 8 up, 8 in 2024-09-21T11:59:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-47", "mode": "writeback"} : dispatch 2024-09-21T11:59:40.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:40.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:40 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:41.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-47"}]': finished 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: osdmap e317: 8 total, 8 up, 8 in 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-47", "mode": "writeback"} : dispatch 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-47"}]': finished 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: osdmap e317: 8 total, 8 up, 8 in 2024-09-21T11:59:41.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-47", "mode": "writeback"} : dispatch 2024-09-21T11:59:41.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1824186697' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:41.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:40 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"} : dispatch 2024-09-21T11:59:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:59:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-47", "mode": "writeback"}]': finished 2024-09-21T11:59:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: osdmap e318: 8 total, 8 up, 8 in 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_count","val": "2"}]': finished 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:41 smithi193 ceph-mon[33333]: osdmap e319: 8 total, 8 up, 8 in 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-47", "mode": "writeback"}]': finished 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: osdmap e318: 8 total, 8 up, 8 in 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:42.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:42.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_count","val": "2"}]': finished 2024-09-21T11:59:42.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:42.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[30293]: osdmap e319: 8 total, 8 up, 8 in 2024-09-21T11:59:42.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:59:42.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-47", "mode": "writeback"}]': finished 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-77452-61"}]': finished 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: osdmap e318: 8 total, 8 up, 8 in 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_count","val": "2"}]': finished 2024-09-21T11:59:42.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:42.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:41 smithi107 ceph-mon[41097]: osdmap e319: 8 total, 8 up, 8 in 2024-09-21T11:59:42.537 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:59:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:42 smithi193 ceph-mon[33333]: pgmap v424: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T11:59:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-77452-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:43.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[41097]: pgmap v424: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-77452-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[30293]: pgmap v424: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-77452-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_period","val": "600"}]': finished 2024-09-21T11:59:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: osdmap e320: 8 total, 8 up, 8 in 2024-09-21T11:59:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T11:59:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:59:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-77452-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T11:59:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: osdmap e321: 8 total, 8 up, 8 in 2024-09-21T11:59:43.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T11:59:44.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: osdmap e320: 8 total, 8 up, 8 in 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-77452-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: osdmap e321: 8 total, 8 up, 8 in 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: osdmap e320: 8 total, 8 up, 8 in 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T11:59:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T11:59:44.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-77452-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:44.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T11:59:44.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: osdmap e321: 8 total, 8 up, 8 in 2024-09-21T11:59:44.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T11:59:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:44 smithi193 ceph-mon[33333]: pgmap v427: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T11:59:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:44 smithi193 ceph-mon[33333]: osdmap e322: 8 total, 8 up, 8 in 2024-09-21T11:59:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-21T11:59:45.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[30293]: pgmap v427: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[30293]: osdmap e322: 8 total, 8 up, 8 in 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[41097]: pgmap v427: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[41097]: osdmap e322: 8 total, 8 up, 8 in 2024-09-21T11:59:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-21T11:59:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:46 smithi193 ceph-mon[33333]: pgmap v430: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-21T11:59:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:46 smithi193 ceph-mon[33333]: osdmap e323: 8 total, 8 up, 8 in 2024-09-21T11:59:46.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-21T11:59:46.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:46.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:46 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:47.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[30293]: pgmap v430: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[30293]: osdmap e323: 8 total, 8 up, 8 in 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[41097]: pgmap v430: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[41097]: osdmap e323: 8 total, 8 up, 8 in 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:46 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-21T11:59:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:47 smithi193 ceph-mon[33333]: osdmap e324: 8 total, 8 up, 8 in 2024-09-21T11:59:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:48.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[41097]: osdmap e324: 8 total, 8 up, 8 in 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[30293]: osdmap e324: 8 total, 8 up, 8 in 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"} : dispatch 2024-09-21T11:59:48.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: pgmap v433: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: osdmap e325: 8 total, 8 up, 8 in 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: pgmap v433: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: osdmap e325: 8 total, 8 up, 8 in 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:49.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:48 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:49.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:59:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: pgmap v433: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:59:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3438677297' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-77452-62"}]': finished 2024-09-21T11:59:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T11:59:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: osdmap e325: 8 total, 8 up, 8 in 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:48 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"}]': finished 2024-09-21T11:59:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: osdmap e326: 8 total, 8 up, 8 in 2024-09-21T11:59:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:50.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"}]': finished 2024-09-21T11:59:50.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:50.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: osdmap e326: 8 total, 8 up, 8 in 2024-09-21T11:59:50.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:50.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:50.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:50.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:50.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"}]': finished 2024-09-21T11:59:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-77452-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: osdmap e326: 8 total, 8 up, 8 in 2024-09-21T11:59:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T11:59:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-47"} : dispatch 2024-09-21T11:59:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:51.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:50 smithi107 ceph-mon[41097]: pgmap v436: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T11:59:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:50 smithi107 ceph-mon[41097]: osdmap e327: 8 total, 8 up, 8 in 2024-09-21T11:59:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:50 smithi107 ceph-mon[30293]: pgmap v436: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T11:59:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:50 smithi107 ceph-mon[30293]: osdmap e327: 8 total, 8 up, 8 in 2024-09-21T11:59:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:50 smithi193 ceph-mon[33333]: pgmap v436: 228 pgs: 228 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T11:59:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:50 smithi193 ceph-mon[33333]: osdmap e327: 8 total, 8 up, 8 in 2024-09-21T11:59:52.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:52.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:51 smithi107 ceph-mon[30293]: osdmap e328: 8 total, 8 up, 8 in 2024-09-21T11:59:52.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:51 smithi107 ceph-mon[41097]: osdmap e328: 8 total, 8 up, 8 in 2024-09-21T11:59:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-77452-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:51 smithi193 ceph-mon[33333]: osdmap e328: 8 total, 8 up, 8 in 2024-09-21T11:59:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-49","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T11:59:52.585 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:11:59:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: pgmap v439: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: osdmap e329: 8 total, 8 up, 8 in 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: osdmap e330: 8 total, 8 up, 8 in 2024-09-21T11:59:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: pgmap v439: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: osdmap e329: 8 total, 8 up, 8 in 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: osdmap e330: 8 total, 8 up, 8 in 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T11:59:53.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:53.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:52 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: pgmap v439: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T11:59:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T11:59:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: osdmap e329: 8 total, 8 up, 8 in 2024-09-21T11:59:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T11:59:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T11:59:53.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: osdmap e330: 8 total, 8 up, 8 in 2024-09-21T11:59:53.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T11:59:53.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:53.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:52 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-49"}]': finished 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: osdmap e331: 8 total, 8 up, 8 in 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-49", "mode": "readproxy"} : dispatch 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-49"}]': finished 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: osdmap e331: 8 total, 8 up, 8 in 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-49", "mode": "readproxy"} : dispatch 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:53 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-49"}]': finished 2024-09-21T11:59:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: osdmap e331: 8 total, 8 up, 8 in 2024-09-21T11:59:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-49", "mode": "readproxy"} : dispatch 2024-09-21T11:59:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2875737272' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:53 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"} : dispatch 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[30293]: pgmap v442: 228 pgs: 17 unknown, 211 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-49", "mode": "readproxy"}]': finished 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[30293]: osdmap e332: 8 total, 8 up, 8 in 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[41097]: pgmap v442: 228 pgs: 17 unknown, 211 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-49", "mode": "readproxy"}]': finished 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:54 smithi107 ceph-mon[41097]: osdmap e332: 8 total, 8 up, 8 in 2024-09-21T11:59:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:54 smithi193 ceph-mon[33333]: pgmap v442: 228 pgs: 17 unknown, 211 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:59:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:54 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T11:59:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-49", "mode": "readproxy"}]': finished 2024-09-21T11:59:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:54 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-77452-63"}]': finished 2024-09-21T11:59:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:54 smithi193 ceph-mon[33333]: osdmap e332: 8 total, 8 up, 8 in 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:56.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:55 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:55 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T11:59:57.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[30293]: pgmap v445: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[30293]: osdmap e333: 8 total, 8 up, 8 in 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[41097]: pgmap v445: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[41097]: osdmap e333: 8 total, 8 up, 8 in 2024-09-21T11:59:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:57.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:56 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:56 smithi193 ceph-mon[33333]: pgmap v445: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:59:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-77452-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T11:59:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:56 smithi193 ceph-mon[33333]: osdmap e333: 8 total, 8 up, 8 in 2024-09-21T11:59:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:56 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[30293]: osdmap e334: 8 total, 8 up, 8 in 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[30293]: osdmap e335: 8 total, 8 up, 8 in 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[41097]: osdmap e334: 8 total, 8 up, 8 in 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T11:59:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:57 smithi107 ceph-mon[41097]: osdmap e335: 8 total, 8 up, 8 in 2024-09-21T11:59:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:57 smithi193 ceph-mon[33333]: osdmap e334: 8 total, 8 up, 8 in 2024-09-21T11:59:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:57 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:59:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:57 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-77452-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T11:59:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:57 smithi193 ceph-mon[33333]: osdmap e335: 8 total, 8 up, 8 in 2024-09-21T11:59:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:58 smithi107 ceph-mon[30293]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T11:59:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:58 smithi107 ceph-mon[30293]: osdmap e336: 8 total, 8 up, 8 in 2024-09-21T11:59:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:58 smithi107 ceph-mon[41097]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T11:59:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:58 smithi107 ceph-mon[41097]: osdmap e336: 8 total, 8 up, 8 in 2024-09-21T11:59:59.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 11:59:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T11:59:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:58 smithi193 ceph-mon[33333]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T11:59:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:58 smithi193 ceph-mon[33333]: osdmap e336: 8 total, 8 up, 8 in 2024-09-21T12:00:00.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 11:59:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 11:59:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 11:59:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: pgmap v451: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: osdmap e337: 8 total, 8 up, 8 in 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 2 pool(s) do not have an application enabled 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: pool 'test-rados-api-smithi107-77755-49' with cache_mode readproxy needs hit_set_type to be set but it is not 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-21T12:00:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: application not enabled on pool 'ExecuteClassPP_smithi107-77452-64' 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[41097]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: pgmap v451: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: osdmap e337: 8 total, 8 up, 8 in 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 2 pool(s) do not have an application enabled 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: pool 'test-rados-api-smithi107-77755-49' with cache_mode readproxy needs hit_set_type to be set but it is not 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: application not enabled on pool 'ExecuteClassPP_smithi107-77452-64' 2024-09-21T12:00:01.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:00 smithi107 ceph-mon[30293]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-21T12:00:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: pgmap v451: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:00:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: osdmap e337: 8 total, 8 up, 8 in 2024-09-21T12:00:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 2 pool(s) do not have an application enabled 2024-09-21T12:00:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-09-21T12:00:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: pool 'test-rados-api-smithi107-77755-49' with cache_mode readproxy needs hit_set_type to be set but it is not 2024-09-21T12:00:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-21T12:00:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-21T12:00:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: application not enabled on pool 'ExecuteClassPP_smithi107-77452-64' 2024-09-21T12:00:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:00 smithi193 ceph-mon[33333]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-21T12:00:02.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T12:00:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[41097]: osdmap e338: 8 total, 8 up, 8 in 2024-09-21T12:00:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T12:00:02.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T12:00:02.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[30293]: osdmap e338: 8 total, 8 up, 8 in 2024-09-21T12:00:02.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:02.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:02.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:01 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T12:00:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:01 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T12:00:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:01 smithi193 ceph-mon[33333]: osdmap e338: 8 total, 8 up, 8 in 2024-09-21T12:00:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1636234771' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:01 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"} : dispatch 2024-09-21T12:00:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:01 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-77452-64"}]': finished 2024-09-21T12:00:02.642 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:00:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: pgmap v454: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:00:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: osdmap e339: 8 total, 8 up, 8 in 2024-09-21T12:00:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:02.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:02.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:02 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:03.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: pgmap v454: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: osdmap e339: 8 total, 8 up, 8 in 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: pgmap v454: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:00:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: osdmap e339: 8 total, 8 up, 8 in 2024-09-21T12:00:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:03.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:02 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:04.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[30293]: osdmap e340: 8 total, 8 up, 8 in 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[41097]: osdmap e340: 8 total, 8 up, 8 in 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:03 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-77452-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:00:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:03 smithi193 ceph-mon[33333]: osdmap e340: 8 total, 8 up, 8 in 2024-09-21T12:00:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:03 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:05.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:04 smithi107 ceph-mon[30293]: pgmap v457: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:04 smithi107 ceph-mon[30293]: osdmap e341: 8 total, 8 up, 8 in 2024-09-21T12:00:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:04 smithi107 ceph-mon[41097]: pgmap v457: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:04 smithi107 ceph-mon[41097]: osdmap e341: 8 total, 8 up, 8 in 2024-09-21T12:00:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:04 smithi193 ceph-mon[33333]: pgmap v457: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:04 smithi193 ceph-mon[33333]: osdmap e341: 8 total, 8 up, 8 in 2024-09-21T12:00:06.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[41097]: osdmap e342: 8 total, 8 up, 8 in 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[41097]: osdmap e343: 8 total, 8 up, 8 in 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[30293]: osdmap e342: 8 total, 8 up, 8 in 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:00:06.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[30293]: osdmap e343: 8 total, 8 up, 8 in 2024-09-21T12:00:06.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T12:00:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:05 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-77452-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:05 smithi193 ceph-mon[33333]: osdmap e342: 8 total, 8 up, 8 in 2024-09-21T12:00:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:00:06.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:05 smithi193 ceph-mon[33333]: osdmap e343: 8 total, 8 up, 8 in 2024-09-21T12:00:06.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T12:00:07.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: pgmap v460: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"}]': finished 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: osdmap e344: 8 total, 8 up, 8 in 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: pgmap v460: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"}]': finished 2024-09-21T12:00:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: osdmap e344: 8 total, 8 up, 8 in 2024-09-21T12:00:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:07.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:06 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: pgmap v460: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:00:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"}]': finished 2024-09-21T12:00:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: osdmap e344: 8 total, 8 up, 8 in 2024-09-21T12:00:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:07.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:06 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2163:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:5d165639:::164:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f43765fc:::165:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b4c720e9:::166:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e694b040:::167:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:afa38db2:::168:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:77ba9f53:::169:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:87495034:::170:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7c96bf0e:::171:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:dbe346cc:::172:head 2024-09-21T12:00:07.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:e943ec24:::173:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f97a9c0c:::174:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:6f26e74d:::175:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4f95e106:::176:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:0e6f2f8f:::177:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:05db05f1:::178:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:38a78d66:::179:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:d095610b:::180:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:a1a9d709:::181:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:1e5d39db:::182:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:f7df4fb9:::183:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:03a7f161:::184:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ba70721e:::185:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:28e5662d:::186:head 2024-09-21T12:00:07.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:973d52de:::187:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:4303eb1c:::188:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:b990b48e:::189:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:29b8165b:::190:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:3547f197:::191:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:7e260936:::192:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:1abec7b1:::193:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:10fdda93:::194:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:15817eea:::195:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:770bab57:::196:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:ed9e13e7:::197:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:71471a8f:::198:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 276:10fb1d02:::199:head 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8029 ms) 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,0 2024-09-21T12:00:07.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first is 1726919962 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,1726919967,1726919968,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,1726919967,1726919968,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,1726919967,1726919968,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,1726919967,1726919968,1726919970,1726919971,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,1726919967,1726919968,1726919970,1726919971,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919962,1726919964,1726919965,1726919967,1726919968,1726919970,1726919971,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726919965,1726919967,1726919968,1726919970,1726919971,1726919973,1726919974,0 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first now 1726919965, trimmed 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21107 ms) 2024-09-21T12:00:07.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-21T12:00:07.672 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: foo0 2024-09-21T12:00:07.672 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-21T12:00:07.672 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14053 ms) 2024-09-21T12:00:07.672 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[30293]: osdmap e345: 8 total, 8 up, 8 in 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:08.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T12:00:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[41097]: osdmap e345: 8 total, 8 up, 8 in 2024-09-21T12:00:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:08.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:07 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-49"} : dispatch 2024-09-21T12:00:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:07 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:07 smithi193 ceph-mon[33333]: osdmap e345: 8 total, 8 up, 8 in 2024-09-21T12:00:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/277576370' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:07 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: pgmap v463: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]} : dispatch 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]}]': finished 2024-09-21T12:00:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]': finished 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]}]': finished 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: osdmap e346: 8 total, 8 up, 8 in 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: pgmap v463: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]} : dispatch 2024-09-21T12:00:09.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:09.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]}]': finished 2024-09-21T12:00:09.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]': finished 2024-09-21T12:00:09.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]}]': finished 2024-09-21T12:00:09.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: osdmap e346: 8 total, 8 up, 8 in 2024-09-21T12:00:09.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:09.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: pgmap v463: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]} : dispatch 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-77452-65"}]': finished 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.b", "id": [3, 7]}]': finished 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]': finished 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.16", "id": [3, 6]}]': finished 2024-09-21T12:00:09.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: osdmap e346: 8 total, 8 up, 8 in 2024-09-21T12:00:09.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-51","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: osdmap e347: 8 total, 8 up, 8 in 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:10.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:10.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:10.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:00:10.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: osdmap e347: 8 total, 8 up, 8 in 2024-09-21T12:00:10.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:09 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-77452-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: osdmap e347: 8 total, 8 up, 8 in 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:10.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:09 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[30293]: pgmap v466: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[30293]: osdmap e348: 8 total, 8 up, 8 in 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[41097]: pgmap v466: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:00:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:00:11.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[41097]: osdmap e348: 8 total, 8 up, 8 in 2024-09-21T12:00:11.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:10 smithi193 ceph-mon[33333]: pgmap v466: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:00:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:00:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:10 smithi193 ceph-mon[33333]: osdmap e348: 8 total, 8 up, 8 in 2024-09-21T12:00:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:12.687 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:00:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[30293]: pgmap v469: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-51"}]': finished 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[30293]: osdmap e349: 8 total, 8 up, 8 in 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-51", "mode": "writeback"} : dispatch 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[41097]: pgmap v469: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:13.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-51"}]': finished 2024-09-21T12:00:13.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[41097]: osdmap e349: 8 total, 8 up, 8 in 2024-09-21T12:00:13.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-51", "mode": "writeback"} : dispatch 2024-09-21T12:00:13.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:12 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:12 smithi193 ceph-mon[33333]: pgmap v469: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:12 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-77452-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-51"}]': finished 2024-09-21T12:00:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:12 smithi193 ceph-mon[33333]: osdmap e349: 8 total, 8 up, 8 in 2024-09-21T12:00:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-51", "mode": "writeback"} : dispatch 2024-09-21T12:00:13.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:12 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:14.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-51", "mode": "writeback"}]': finished 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[30293]: osdmap e350: 8 total, 8 up, 8 in 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-51", "mode": "writeback"}]': finished 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[41097]: osdmap e350: 8 total, 8 up, 8 in 2024-09-21T12:00:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:00:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:13 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:00:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-51", "mode": "writeback"}]': finished 2024-09-21T12:00:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:13 smithi193 ceph-mon[33333]: osdmap e350: 8 total, 8 up, 8 in 2024-09-21T12:00:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[30293]: pgmap v472: 236 pgs: 21 unknown, 215 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s; 16 B/s, 7 objects/s recovering 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[30293]: osdmap e351: 8 total, 8 up, 8 in 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[41097]: pgmap v472: 236 pgs: 21 unknown, 215 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s; 16 B/s, 7 objects/s recovering 2024-09-21T12:00:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:00:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[41097]: osdmap e351: 8 total, 8 up, 8 in 2024-09-21T12:00:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:00:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:15.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:14 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:14 smithi193 ceph-mon[33333]: pgmap v472: 236 pgs: 21 unknown, 215 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s; 16 B/s, 7 objects/s recovering 2024-09-21T12:00:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:00:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:14 smithi193 ceph-mon[33333]: osdmap e351: 8 total, 8 up, 8 in 2024-09-21T12:00:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:00:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:15.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:14 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3021 ms) 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7024 ms) 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7026 ms) 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7032 ms) 2024-09-21T12:00:15.711 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7051 ms) 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7037 ms) 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7034 ms) 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139342 ms total) 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (286065 ms total) 2024-09-21T12:00:15.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: osdmap e352: 8 total, 8 up, 8 in 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: osdmap e352: 8 total, 8 up, 8 in 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:00:16.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:16.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:15 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:00:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:00:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: osdmap e352: 8 total, 8 up, 8 in 2024-09-21T12:00:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/4143597803' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:00:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"} : dispatch 2024-09-21T12:00:16.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:15 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[30293]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s; 38 B/s, 15 objects/s recovering 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[30293]: osdmap e353: 8 total, 8 up, 8 in 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[41097]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s; 38 B/s, 15 objects/s recovering 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[41097]: osdmap e353: 8 total, 8 up, 8 in 2024-09-21T12:00:17.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:00:17.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:00:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:16 smithi193 ceph-mon[33333]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s; 38 B/s, 15 objects/s recovering 2024-09-21T12:00:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:00:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-77452-66"}]': finished 2024-09-21T12:00:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:16 smithi193 ceph-mon[33333]: osdmap e353: 8 total, 8 up, 8 in 2024-09-21T12:00:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:00:17.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:00:18.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:17 smithi107 ceph-mon[30293]: osdmap e354: 8 total, 8 up, 8 in 2024-09-21T12:00:18.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-21T12:00:18.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "target_max_objects","val": "1"}]': finished 2024-09-21T12:00:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:17 smithi107 ceph-mon[41097]: osdmap e354: 8 total, 8 up, 8 in 2024-09-21T12:00:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-21T12:00:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "target_max_objects","val": "1"}]': finished 2024-09-21T12:00:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:17 smithi193 ceph-mon[33333]: osdmap e354: 8 total, 8 up, 8 in 2024-09-21T12:00:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "target_max_objects","val": "1"} : dispatch 2024-09-21T12:00:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-51","var": "target_max_objects","val": "1"}]': finished 2024-09-21T12:00:19.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:18 smithi107 ceph-mon[41097]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 255 B/s wr, 1 op/s; 21 B/s, 8 objects/s recovering 2024-09-21T12:00:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:18 smithi107 ceph-mon[41097]: osdmap e355: 8 total, 8 up, 8 in 2024-09-21T12:00:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:18 smithi107 ceph-mon[30293]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 255 B/s wr, 1 op/s; 21 B/s, 8 objects/s recovering 2024-09-21T12:00:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:18 smithi107 ceph-mon[30293]: osdmap e355: 8 total, 8 up, 8 in 2024-09-21T12:00:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:18 smithi193 ceph-mon[33333]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 255 B/s wr, 1 op/s; 21 B/s, 8 objects/s recovering 2024-09-21T12:00:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:18 smithi193 ceph-mon[33333]: osdmap e355: 8 total, 8 up, 8 in 2024-09-21T12:00:19.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:00:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:00:20.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:21.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:20 smithi107 ceph-mon[41097]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:20 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-21T12:00:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:20 smithi107 ceph-mon[30293]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:20 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-21T12:00:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:20 smithi193 ceph-mon[33333]: pgmap v480: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:20 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-21T12:00:22.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:00:22.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:00:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:00:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:00:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:00:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:00:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:21 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:00:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:00:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:21 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:00:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:21 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:22.733 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:00:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:00:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:23.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:22 smithi107 ceph-mon[41097]: pgmap v481: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:22 smithi107 ceph-mon[30293]: pgmap v481: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:22 smithi193 ceph-mon[33333]: pgmap v481: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:24.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:23 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:23 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:23 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:25.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:24 smithi107 ceph-mon[30293]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:24 smithi107 ceph-mon[41097]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:24 smithi193 ceph-mon[33333]: pgmap v482: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:27.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:27 smithi107 ceph-mon[30293]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:27 smithi107 ceph-mon[41097]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:27 smithi193 ceph-mon[33333]: pgmap v483: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:28.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:29.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:00:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:00:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:29 smithi193 ceph-mon[33333]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:00:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:29 smithi193 ceph-mon[33333]: osdmap e356: 8 total, 8 up, 8 in 2024-09-21T12:00:29.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:29.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[30293]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[30293]: osdmap e356: 8 total, 8 up, 8 in 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[41097]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[41097]: osdmap e356: 8 total, 8 up, 8 in 2024-09-21T12:00:29.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:30 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"}]': finished 2024-09-21T12:00:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:30 smithi193 ceph-mon[33333]: osdmap e357: 8 total, 8 up, 8 in 2024-09-21T12:00:30.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:30.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"}]': finished 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[30293]: osdmap e357: 8 total, 8 up, 8 in 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"}]': finished 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[41097]: osdmap e357: 8 total, 8 up, 8 in 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:30.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-51"} : dispatch 2024-09-21T12:00:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:31 smithi193 ceph-mon[33333]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:00:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:31 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-21T12:00:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:31 smithi193 ceph-mon[33333]: osdmap e358: 8 total, 8 up, 8 in 2024-09-21T12:00:31.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:31 smithi107 ceph-mon[30293]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:00:31.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:31 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-21T12:00:31.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:31 smithi107 ceph-mon[30293]: osdmap e358: 8 total, 8 up, 8 in 2024-09-21T12:00:31.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:31 smithi107 ceph-mon[41097]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:00:31.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:31 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-21T12:00:31.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:31 smithi107 ceph-mon[41097]: osdmap e358: 8 total, 8 up, 8 in 2024-09-21T12:00:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:32 smithi193 ceph-mon[33333]: osdmap e359: 8 total, 8 up, 8 in 2024-09-21T12:00:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:32.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:32 smithi107 ceph-mon[41097]: osdmap e359: 8 total, 8 up, 8 in 2024-09-21T12:00:32.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:32.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:00:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:00:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:32.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:32 smithi107 ceph-mon[30293]: osdmap e359: 8 total, 8 up, 8 in 2024-09-21T12:00:32.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-53","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:33 smithi193 ceph-mon[33333]: pgmap v490: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-21T12:00:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:33 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:33 smithi193 ceph-mon[33333]: osdmap e360: 8 total, 8 up, 8 in 2024-09-21T12:00:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:33.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-53"} : dispatch 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[41097]: pgmap v490: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[41097]: osdmap e360: 8 total, 8 up, 8 in 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-53"} : dispatch 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[30293]: pgmap v490: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:33.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[30293]: osdmap e360: 8 total, 8 up, 8 in 2024-09-21T12:00:33.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:33.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-53"} : dispatch 2024-09-21T12:00:34.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:34 smithi107 ceph-mon[30293]: osdmap e361: 8 total, 8 up, 8 in 2024-09-21T12:00:34.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:34 smithi107 ceph-mon[41097]: osdmap e361: 8 total, 8 up, 8 in 2024-09-21T12:00:34.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:34 smithi193 ceph-mon[33333]: osdmap e361: 8 total, 8 up, 8 in 2024-09-21T12:00:35.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:35 smithi107 ceph-mon[30293]: pgmap v493: 196 pgs: 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:35 smithi107 ceph-mon[30293]: osdmap e362: 8 total, 8 up, 8 in 2024-09-21T12:00:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:35 smithi107 ceph-mon[41097]: pgmap v493: 196 pgs: 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:35 smithi107 ceph-mon[41097]: osdmap e362: 8 total, 8 up, 8 in 2024-09-21T12:00:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:35 smithi193 ceph-mon[33333]: pgmap v493: 196 pgs: 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:35 smithi193 ceph-mon[33333]: osdmap e362: 8 total, 8 up, 8 in 2024-09-21T12:00:35.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-55","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[30293]: osdmap e363: 8 total, 8 up, 8 in 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-55"} : dispatch 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[41097]: osdmap e363: 8 total, 8 up, 8 in 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:36.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:36.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-55"} : dispatch 2024-09-21T12:00:36.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:36.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:36 smithi193 ceph-mon[33333]: osdmap e363: 8 total, 8 up, 8 in 2024-09-21T12:00:36.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:36.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:36.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-55"} : dispatch 2024-09-21T12:00:37.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:37 smithi107 ceph-mon[41097]: pgmap v496: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:37.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:37 smithi107 ceph-mon[41097]: osdmap e364: 8 total, 8 up, 8 in 2024-09-21T12:00:37.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:37 smithi107 ceph-mon[30293]: pgmap v496: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:37.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:37 smithi107 ceph-mon[30293]: osdmap e364: 8 total, 8 up, 8 in 2024-09-21T12:00:37.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:37 smithi193 ceph-mon[33333]: pgmap v496: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:37.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:37 smithi193 ceph-mon[33333]: osdmap e364: 8 total, 8 up, 8 in 2024-09-21T12:00:38.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[30293]: osdmap e365: 8 total, 8 up, 8 in 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[41097]: osdmap e365: 8 total, 8 up, 8 in 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:38 smithi193 ceph-mon[33333]: osdmap e365: 8 total, 8 up, 8 in 2024-09-21T12:00:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-57","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:38 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:38.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:38 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:00:38.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:39.050 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:00:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[30293]: pgmap v499: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[30293]: osdmap e366: 8 total, 8 up, 8 in 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-57"} : dispatch 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[41097]: pgmap v499: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:39.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[41097]: osdmap e366: 8 total, 8 up, 8 in 2024-09-21T12:00:39.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:39.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-57"} : dispatch 2024-09-21T12:00:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:39 smithi193 ceph-mon[33333]: pgmap v499: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:00:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:39.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:39 smithi193 ceph-mon[33333]: osdmap e366: 8 total, 8 up, 8 in 2024-09-21T12:00:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:39.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-57"} : dispatch 2024-09-21T12:00:40.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:40 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:40.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:40 smithi107 ceph-mon[30293]: osdmap e367: 8 total, 8 up, 8 in 2024-09-21T12:00:40.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:40 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:40.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:40 smithi107 ceph-mon[41097]: osdmap e367: 8 total, 8 up, 8 in 2024-09-21T12:00:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:40 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:40 smithi193 ceph-mon[33333]: osdmap e367: 8 total, 8 up, 8 in 2024-09-21T12:00:41.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:41 smithi107 ceph-mon[30293]: pgmap v502: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:00:41.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:41 smithi107 ceph-mon[30293]: osdmap e368: 8 total, 8 up, 8 in 2024-09-21T12:00:41.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:41.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:41 smithi107 ceph-mon[41097]: pgmap v502: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:00:41.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:41 smithi107 ceph-mon[41097]: osdmap e368: 8 total, 8 up, 8 in 2024-09-21T12:00:41.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:41 smithi193 ceph-mon[33333]: pgmap v502: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:00:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:41 smithi193 ceph-mon[33333]: osdmap e368: 8 total, 8 up, 8 in 2024-09-21T12:00:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-59","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:42.505 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:00:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[30293]: osdmap e369: 8 total, 8 up, 8 in 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-59"} : dispatch 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[41097]: osdmap e369: 8 total, 8 up, 8 in 2024-09-21T12:00:42.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:42.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-59"} : dispatch 2024-09-21T12:00:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:42 smithi193 ceph-mon[33333]: osdmap e369: 8 total, 8 up, 8 in 2024-09-21T12:00:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:42.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-59"} : dispatch 2024-09-21T12:00:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:43 smithi107 ceph-mon[41097]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:00:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:43 smithi107 ceph-mon[41097]: osdmap e370: 8 total, 8 up, 8 in 2024-09-21T12:00:43.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:43 smithi107 ceph-mon[30293]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:00:43.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:43 smithi107 ceph-mon[30293]: osdmap e370: 8 total, 8 up, 8 in 2024-09-21T12:00:43.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:43 smithi193 ceph-mon[33333]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:00:43.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:43 smithi193 ceph-mon[33333]: osdmap e370: 8 total, 8 up, 8 in 2024-09-21T12:00:44.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:44 smithi107 ceph-mon[30293]: osdmap e371: 8 total, 8 up, 8 in 2024-09-21T12:00:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:44.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:44 smithi107 ceph-mon[30293]: pgmap v508: 228 pgs: 12 creating+peering, 20 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:44 smithi107 ceph-mon[41097]: osdmap e371: 8 total, 8 up, 8 in 2024-09-21T12:00:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:44.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:44 smithi107 ceph-mon[41097]: pgmap v508: 228 pgs: 12 creating+peering, 20 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:44 smithi193 ceph-mon[33333]: osdmap e371: 8 total, 8 up, 8 in 2024-09-21T12:00:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-61","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:44 smithi193 ceph-mon[33333]: pgmap v508: 228 pgs: 12 creating+peering, 20 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:00:45.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[30293]: osdmap e372: 8 total, 8 up, 8 in 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[41097]: osdmap e372: 8 total, 8 up, 8 in 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:45.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:00:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:45 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:45 smithi193 ceph-mon[33333]: osdmap e372: 8 total, 8 up, 8 in 2024-09-21T12:00:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:00:46.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[30293]: osdmap e373: 8 total, 8 up, 8 in 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-61"} : dispatch 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[30293]: pgmap v511: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[41097]: osdmap e373: 8 total, 8 up, 8 in 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-61"} : dispatch 2024-09-21T12:00:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:46 smithi107 ceph-mon[41097]: pgmap v511: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:00:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:00:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:46 smithi193 ceph-mon[33333]: osdmap e373: 8 total, 8 up, 8 in 2024-09-21T12:00:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:00:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-61"} : dispatch 2024-09-21T12:00:46.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:46 smithi193 ceph-mon[33333]: pgmap v511: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:00:47.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:47 smithi107 ceph-mon[30293]: osdmap e374: 8 total, 8 up, 8 in 2024-09-21T12:00:47.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:47 smithi107 ceph-mon[41097]: osdmap e374: 8 total, 8 up, 8 in 2024-09-21T12:00:47.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:47 smithi193 ceph-mon[33333]: osdmap e374: 8 total, 8 up, 8 in 2024-09-21T12:00:48.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:48 smithi107 ceph-mon[30293]: osdmap e375: 8 total, 8 up, 8 in 2024-09-21T12:00:48.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:48.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:48 smithi107 ceph-mon[30293]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 1 op/s 2024-09-21T12:00:48.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:48 smithi107 ceph-mon[41097]: osdmap e375: 8 total, 8 up, 8 in 2024-09-21T12:00:48.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:48.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:48 smithi107 ceph-mon[41097]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 1 op/s 2024-09-21T12:00:48.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:48 smithi193 ceph-mon[33333]: osdmap e375: 8 total, 8 up, 8 in 2024-09-21T12:00:48.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-63","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:00:48.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:48 smithi193 ceph-mon[33333]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 1 op/s 2024-09-21T12:00:49.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:00:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:00:49.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[30293]: osdmap e376: 8 total, 8 up, 8 in 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[41097]: osdmap e376: 8 total, 8 up, 8 in 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:00:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:00:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:49 smithi193 ceph-mon[33333]: osdmap e376: 8 total, 8 up, 8 in 2024-09-21T12:00:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:00:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:00:49.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:50.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:00:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:50 smithi107 ceph-mon[30293]: osdmap e377: 8 total, 8 up, 8 in 2024-09-21T12:00:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:50 smithi107 ceph-mon[30293]: pgmap v517: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:50.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:00:50.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:50 smithi107 ceph-mon[41097]: osdmap e377: 8 total, 8 up, 8 in 2024-09-21T12:00:50.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:50 smithi107 ceph-mon[41097]: pgmap v517: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:50.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:00:50.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:50 smithi193 ceph-mon[33333]: osdmap e377: 8 total, 8 up, 8 in 2024-09-21T12:00:50.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:50 smithi193 ceph-mon[33333]: pgmap v517: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:51.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:51 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:51.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:51 smithi107 ceph-mon[30293]: osdmap e378: 8 total, 8 up, 8 in 2024-09-21T12:00:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:51 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:51 smithi107 ceph-mon[41097]: osdmap e378: 8 total, 8 up, 8 in 2024-09-21T12:00:51.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:51 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:00:51.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:51 smithi193 ceph-mon[33333]: osdmap e378: 8 total, 8 up, 8 in 2024-09-21T12:00:52.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:52 smithi107 ceph-mon[41097]: osdmap e379: 8 total, 8 up, 8 in 2024-09-21T12:00:52.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:52 smithi107 ceph-mon[41097]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:52.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:52 smithi107 ceph-mon[30293]: osdmap e379: 8 total, 8 up, 8 in 2024-09-21T12:00:52.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:52 smithi107 ceph-mon[30293]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:52.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:00:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:00:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:52.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:52 smithi193 ceph-mon[33333]: osdmap e379: 8 total, 8 up, 8 in 2024-09-21T12:00:52.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:52 smithi193 ceph-mon[33333]: pgmap v520: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:00:53.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:53 smithi107 ceph-mon[41097]: osdmap e380: 8 total, 8 up, 8 in 2024-09-21T12:00:53.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:53.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:53 smithi107 ceph-mon[30293]: osdmap e380: 8 total, 8 up, 8 in 2024-09-21T12:00:53.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:53 smithi193 ceph-mon[33333]: osdmap e380: 8 total, 8 up, 8 in 2024-09-21T12:00:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:54.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:54 smithi107 ceph-mon[30293]: osdmap e381: 8 total, 8 up, 8 in 2024-09-21T12:00:54.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:54 smithi107 ceph-mon[30293]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-21T12:00:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:54 smithi107 ceph-mon[41097]: osdmap e381: 8 total, 8 up, 8 in 2024-09-21T12:00:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:54 smithi107 ceph-mon[41097]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-21T12:00:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:54 smithi193 ceph-mon[33333]: osdmap e381: 8 total, 8 up, 8 in 2024-09-21T12:00:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:54 smithi193 ceph-mon[33333]: pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-21T12:00:56.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:56 smithi107 ceph-mon[30293]: pgmap v524: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-21T12:00:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:56 smithi107 ceph-mon[41097]: pgmap v524: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-21T12:00:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:56 smithi193 ceph-mon[33333]: pgmap v524: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-09-21T12:00:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:58 smithi107 ceph-mon[30293]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-21T12:00:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:58 smithi107 ceph-mon[41097]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-21T12:00:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:58 smithi193 ceph-mon[33333]: pgmap v525: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-21T12:00:59.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:00:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:00:59.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:00:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:59.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:00:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:00:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:00.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:00 smithi107 ceph-mon[41097]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-21T12:01:00.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:00 smithi107 ceph-mon[30293]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-21T12:01:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:00 smithi193 ceph-mon[33333]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-21T12:01:02.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:01:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:01:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:02 smithi107 ceph-mon[30293]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:01:02.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:02 smithi107 ceph-mon[30293]: osdmap e382: 8 total, 8 up, 8 in 2024-09-21T12:01:02.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:02 smithi107 ceph-mon[41097]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:01:02.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:02 smithi107 ceph-mon[41097]: osdmap e382: 8 total, 8 up, 8 in 2024-09-21T12:01:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:02 smithi193 ceph-mon[33333]: pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:01:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:02 smithi193 ceph-mon[33333]: osdmap e382: 8 total, 8 up, 8 in 2024-09-21T12:01:04.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:04 smithi107 ceph-mon[30293]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 818 B/s wr, 2 op/s 2024-09-21T12:01:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:04 smithi107 ceph-mon[30293]: osdmap e383: 8 total, 8 up, 8 in 2024-09-21T12:01:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:04 smithi107 ceph-mon[41097]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 818 B/s wr, 2 op/s 2024-09-21T12:01:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:04 smithi107 ceph-mon[41097]: osdmap e383: 8 total, 8 up, 8 in 2024-09-21T12:01:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:04 smithi193 ceph-mon[33333]: pgmap v529: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 818 B/s wr, 2 op/s 2024-09-21T12:01:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:04 smithi193 ceph-mon[33333]: osdmap e383: 8 total, 8 up, 8 in 2024-09-21T12:01:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:06 smithi107 ceph-mon[30293]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:01:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:06 smithi107 ceph-mon[41097]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:01:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:06 smithi193 ceph-mon[33333]: pgmap v531: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:01:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:08 smithi107 ceph-mon[41097]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:08 smithi107 ceph-mon[30293]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:08 smithi193 ceph-mon[33333]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:09.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:01:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:01:09.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:09.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:10.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:10 smithi107 ceph-mon[41097]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:10.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:10 smithi107 ceph-mon[30293]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:10 smithi193 ceph-mon[33333]: pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:12 smithi107 ceph-mon[30293]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:12.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:12 smithi107 ceph-mon[30293]: osdmap e384: 8 total, 8 up, 8 in 2024-09-21T12:01:12.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:01:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:01:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:12 smithi107 ceph-mon[41097]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:12.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:12 smithi107 ceph-mon[41097]: osdmap e384: 8 total, 8 up, 8 in 2024-09-21T12:01:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:12 smithi193 ceph-mon[33333]: pgmap v534: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:12 smithi193 ceph-mon[33333]: osdmap e384: 8 total, 8 up, 8 in 2024-09-21T12:01:14.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:14 smithi107 ceph-mon[41097]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:14.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:14 smithi107 ceph-mon[41097]: osdmap e385: 8 total, 8 up, 8 in 2024-09-21T12:01:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:14 smithi107 ceph-mon[30293]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:14.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:14 smithi107 ceph-mon[30293]: osdmap e385: 8 total, 8 up, 8 in 2024-09-21T12:01:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:14 smithi193 ceph-mon[33333]: pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:14.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:14 smithi193 ceph-mon[33333]: osdmap e385: 8 total, 8 up, 8 in 2024-09-21T12:01:16.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:16 smithi107 ceph-mon[30293]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:16.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:16 smithi107 ceph-mon[41097]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:16 smithi193 ceph-mon[33333]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:01:18.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:18 smithi193 ceph-mon[33333]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:18 smithi107 ceph-mon[30293]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:18 smithi107 ceph-mon[41097]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:19.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:01:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:01:19.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:20.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:20.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:20 smithi193 ceph-mon[33333]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:01:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:20 smithi107 ceph-mon[30293]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:01:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:20 smithi107 ceph-mon[41097]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:01:22.518 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:01:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:22 smithi193 ceph-mon[33333]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:22 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:22 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:22 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:22 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:01:22.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:22 smithi193 ceph-mon[33333]: osdmap e386: 8 total, 8 up, 8 in 2024-09-21T12:01:23.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[30293]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[30293]: osdmap e386: 8 total, 8 up, 8 in 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[41097]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:01:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:22 smithi107 ceph-mon[41097]: osdmap e386: 8 total, 8 up, 8 in 2024-09-21T12:01:23.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:24.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:24 smithi193 ceph-mon[33333]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-21T12:01:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-63"} : dispatch 2024-09-21T12:01:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:24 smithi107 ceph-mon[30293]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-21T12:01:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-63"} : dispatch 2024-09-21T12:01:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:24 smithi107 ceph-mon[41097]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-21T12:01:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-63"} : dispatch 2024-09-21T12:01:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:25 smithi193 ceph-mon[33333]: osdmap e387: 8 total, 8 up, 8 in 2024-09-21T12:01:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:25 smithi193 ceph-mon[33333]: osdmap e388: 8 total, 8 up, 8 in 2024-09-21T12:01:25.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:26.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:25 smithi107 ceph-mon[41097]: osdmap e387: 8 total, 8 up, 8 in 2024-09-21T12:01:26.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:25 smithi107 ceph-mon[41097]: osdmap e388: 8 total, 8 up, 8 in 2024-09-21T12:01:26.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:26.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:25 smithi107 ceph-mon[30293]: osdmap e387: 8 total, 8 up, 8 in 2024-09-21T12:01:26.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:25 smithi107 ceph-mon[30293]: osdmap e388: 8 total, 8 up, 8 in 2024-09-21T12:01:26.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-65","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:26 smithi193 ceph-mon[33333]: pgmap v545: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:26 smithi193 ceph-mon[33333]: osdmap e389: 8 total, 8 up, 8 in 2024-09-21T12:01:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:26.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:01:27.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[41097]: pgmap v545: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[41097]: osdmap e389: 8 total, 8 up, 8 in 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[30293]: pgmap v545: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[30293]: osdmap e389: 8 total, 8 up, 8 in 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:01:28.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:28 smithi193 ceph-mon[33333]: pgmap v548: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:28.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:01:28.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:28 smithi193 ceph-mon[33333]: osdmap e390: 8 total, 8 up, 8 in 2024-09-21T12:01:29.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:28 smithi107 ceph-mon[41097]: pgmap v548: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:01:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:28 smithi107 ceph-mon[41097]: osdmap e390: 8 total, 8 up, 8 in 2024-09-21T12:01:29.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:28 smithi107 ceph-mon[30293]: pgmap v548: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:29.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:01:29.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:28 smithi107 ceph-mon[30293]: osdmap e390: 8 total, 8 up, 8 in 2024-09-21T12:01:29.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:01:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:01:30.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:29 smithi107 ceph-mon[41097]: osdmap e391: 8 total, 8 up, 8 in 2024-09-21T12:01:30.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:30.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:29 smithi107 ceph-mon[30293]: osdmap e391: 8 total, 8 up, 8 in 2024-09-21T12:01:30.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:29 smithi193 ceph-mon[33333]: osdmap e391: 8 total, 8 up, 8 in 2024-09-21T12:01:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:30 smithi107 ceph-mon[41097]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T12:01:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:30 smithi107 ceph-mon[41097]: osdmap e392: 8 total, 8 up, 8 in 2024-09-21T12:01:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:30 smithi107 ceph-mon[30293]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T12:01:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:30 smithi107 ceph-mon[30293]: osdmap e392: 8 total, 8 up, 8 in 2024-09-21T12:01:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:30 smithi193 ceph-mon[33333]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T12:01:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:30 smithi193 ceph-mon[33333]: osdmap e392: 8 total, 8 up, 8 in 2024-09-21T12:01:32.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:31 smithi107 ceph-mon[41097]: osdmap e393: 8 total, 8 up, 8 in 2024-09-21T12:01:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:31 smithi107 ceph-mon[30293]: osdmap e393: 8 total, 8 up, 8 in 2024-09-21T12:01:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:31 smithi193 ceph-mon[33333]: osdmap e393: 8 total, 8 up, 8 in 2024-09-21T12:01:32.572 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:01:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:01:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:32.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:32 smithi193 ceph-mon[33333]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T12:01:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:32 smithi107 ceph-mon[30293]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T12:01:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:32 smithi107 ceph-mon[41097]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-21T12:01:35.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:34 smithi107 ceph-mon[41097]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-21T12:01:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:34 smithi107 ceph-mon[30293]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-21T12:01:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:34 smithi193 ceph-mon[33333]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-21T12:01:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:36 smithi107 ceph-mon[30293]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T12:01:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:36 smithi107 ceph-mon[41097]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T12:01:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:36 smithi193 ceph-mon[33333]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-21T12:01:38.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:38 smithi107 ceph-mon[30293]: osdmap e394: 8 total, 8 up, 8 in 2024-09-21T12:01:38.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:38 smithi107 ceph-mon[30293]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 521 B/s wr, 3 op/s 2024-09-21T12:01:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:38 smithi107 ceph-mon[41097]: osdmap e394: 8 total, 8 up, 8 in 2024-09-21T12:01:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:38 smithi107 ceph-mon[41097]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 521 B/s wr, 3 op/s 2024-09-21T12:01:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:38 smithi193 ceph-mon[33333]: osdmap e394: 8 total, 8 up, 8 in 2024-09-21T12:01:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:38 smithi193 ceph-mon[33333]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 521 B/s wr, 3 op/s 2024-09-21T12:01:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:39.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:01:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:01:39.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:40.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:40 smithi107 ceph-mon[30293]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 463 B/s wr, 3 op/s 2024-09-21T12:01:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:40 smithi107 ceph-mon[41097]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 463 B/s wr, 3 op/s 2024-09-21T12:01:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:40 smithi193 ceph-mon[33333]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 463 B/s wr, 3 op/s 2024-09-21T12:01:41.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:41.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-65"} : dispatch 2024-09-21T12:01:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-65"} : dispatch 2024-09-21T12:01:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-65"} : dispatch 2024-09-21T12:01:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:01:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[30293]: osdmap e395: 8 total, 8 up, 8 in 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[30293]: pgmap v561: 196 pgs: 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[30293]: osdmap e396: 8 total, 8 up, 8 in 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[41097]: osdmap e395: 8 total, 8 up, 8 in 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[41097]: pgmap v561: 196 pgs: 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[41097]: osdmap e396: 8 total, 8 up, 8 in 2024-09-21T12:01:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:42 smithi193 ceph-mon[33333]: osdmap e395: 8 total, 8 up, 8 in 2024-09-21T12:01:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:42 smithi193 ceph-mon[33333]: pgmap v561: 196 pgs: 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:42 smithi193 ceph-mon[33333]: osdmap e396: 8 total, 8 up, 8 in 2024-09-21T12:01:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-67","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:44 smithi193 ceph-mon[33333]: osdmap e397: 8 total, 8 up, 8 in 2024-09-21T12:01:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:44.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:44 smithi193 ceph-mon[33333]: pgmap v564: 228 pgs: 14 creating+activating, 16 unknown, 198 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:44.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:44.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[30293]: osdmap e397: 8 total, 8 up, 8 in 2024-09-21T12:01:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[30293]: pgmap v564: 228 pgs: 14 creating+activating, 16 unknown, 198 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[41097]: osdmap e397: 8 total, 8 up, 8 in 2024-09-21T12:01:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:44 smithi107 ceph-mon[41097]: pgmap v564: 228 pgs: 14 creating+activating, 16 unknown, 198 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:01:45.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:45 smithi107 ceph-mon[30293]: osdmap e398: 8 total, 8 up, 8 in 2024-09-21T12:01:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-67"} : dispatch 2024-09-21T12:01:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:45 smithi107 ceph-mon[41097]: osdmap e398: 8 total, 8 up, 8 in 2024-09-21T12:01:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-67"} : dispatch 2024-09-21T12:01:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:45 smithi193 ceph-mon[33333]: osdmap e398: 8 total, 8 up, 8 in 2024-09-21T12:01:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-67"} : dispatch 2024-09-21T12:01:46.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[30293]: osdmap e399: 8 total, 8 up, 8 in 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[30293]: pgmap v567: 196 pgs: 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[30293]: osdmap e400: 8 total, 8 up, 8 in 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[41097]: osdmap e399: 8 total, 8 up, 8 in 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[41097]: pgmap v567: 196 pgs: 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[41097]: osdmap e400: 8 total, 8 up, 8 in 2024-09-21T12:01:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:46 smithi193 ceph-mon[33333]: osdmap e399: 8 total, 8 up, 8 in 2024-09-21T12:01:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:46 smithi193 ceph-mon[33333]: pgmap v567: 196 pgs: 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:01:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:46 smithi193 ceph-mon[33333]: osdmap e400: 8 total, 8 up, 8 in 2024-09-21T12:01:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-69","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[41097]: osdmap e401: 8 total, 8 up, 8 in 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-69"} : dispatch 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[41097]: pgmap v570: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[30293]: osdmap e401: 8 total, 8 up, 8 in 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-69"} : dispatch 2024-09-21T12:01:48.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:48 smithi107 ceph-mon[30293]: pgmap v570: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T12:01:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-mon[33333]: osdmap e401: 8 total, 8 up, 8 in 2024-09-21T12:01:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:01:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-69"} : dispatch 2024-09-21T12:01:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-mon[33333]: pgmap v570: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T12:01:49.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:01:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:01:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:49 smithi107 ceph-mon[30293]: osdmap e402: 8 total, 8 up, 8 in 2024-09-21T12:01:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:49 smithi107 ceph-mon[41097]: osdmap e402: 8 total, 8 up, 8 in 2024-09-21T12:01:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:49 smithi193 ceph-mon[33333]: osdmap e402: 8 total, 8 up, 8 in 2024-09-21T12:01:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:50.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:50 smithi107 ceph-mon[41097]: osdmap e403: 8 total, 8 up, 8 in 2024-09-21T12:01:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:50 smithi107 ceph-mon[41097]: pgmap v573: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:01:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:50 smithi107 ceph-mon[30293]: osdmap e403: 8 total, 8 up, 8 in 2024-09-21T12:01:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:50 smithi107 ceph-mon[30293]: pgmap v573: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:01:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:50 smithi193 ceph-mon[33333]: osdmap e403: 8 total, 8 up, 8 in 2024-09-21T12:01:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-71","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:01:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:50 smithi193 ceph-mon[33333]: pgmap v573: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:01:51.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[41097]: osdmap e404: 8 total, 8 up, 8 in 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[30293]: osdmap e404: 8 total, 8 up, 8 in 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:01:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:51 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:01:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:01:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:51 smithi193 ceph-mon[33333]: osdmap e404: 8 total, 8 up, 8 in 2024-09-21T12:01:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:01:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:01:52.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:01:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-mon[30293]: osdmap e405: 8 total, 8 up, 8 in 2024-09-21T12:01:52.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-mon[30293]: pgmap v576: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:01:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:01:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-mon[41097]: osdmap e405: 8 total, 8 up, 8 in 2024-09-21T12:01:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-mon[41097]: pgmap v576: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:01:52.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:01:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:01:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:01:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:52 smithi193 ceph-mon[33333]: osdmap e405: 8 total, 8 up, 8 in 2024-09-21T12:01:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:52 smithi193 ceph-mon[33333]: pgmap v576: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:01:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:53 smithi107 ceph-mon[30293]: osdmap e406: 8 total, 8 up, 8 in 2024-09-21T12:01:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:53 smithi107 ceph-mon[41097]: osdmap e406: 8 total, 8 up, 8 in 2024-09-21T12:01:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:53 smithi193 ceph-mon[33333]: osdmap e406: 8 total, 8 up, 8 in 2024-09-21T12:01:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:54.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:54 smithi107 ceph-mon[30293]: pgmap v578: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-21T12:01:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:54 smithi107 ceph-mon[41097]: pgmap v578: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-21T12:01:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:54 smithi193 ceph-mon[33333]: pgmap v578: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-21T12:01:56.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:56 smithi107 ceph-mon[30293]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-21T12:01:56.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:56 smithi107 ceph-mon[41097]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-21T12:01:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:56 smithi193 ceph-mon[33333]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-21T12:01:57.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:57 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:01:57.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:57 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:01:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:57 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:01:58.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:58 smithi107 ceph-mon[30293]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 875 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-21T12:01:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:58 smithi107 ceph-mon[41097]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 875 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-21T12:01:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:58 smithi193 ceph-mon[33333]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 875 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-21T12:01:59.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:01:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:01:59.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:01:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:59.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:01:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:01:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:00.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:00 smithi107 ceph-mon[30293]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-21T12:02:00.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:00 smithi107 ceph-mon[41097]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-21T12:02:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:00 smithi193 ceph-mon[33333]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-09-21T12:02:02.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:02 smithi107 ceph-mon[30293]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T12:02:02.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:02:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:02:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:02:02.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:02 smithi107 ceph-mon[41097]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T12:02:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:02 smithi193 ceph-mon[33333]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T12:02:04.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:04 smithi107 ceph-mon[30293]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-21T12:02:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:04 smithi107 ceph-mon[41097]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-21T12:02:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:04 smithi193 ceph-mon[33333]: pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-21T12:02:06.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:06 smithi107 ceph-mon[30293]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T12:02:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:06 smithi107 ceph-mon[41097]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T12:02:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:06 smithi193 ceph-mon[33333]: pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T12:02:08.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:08 smithi107 ceph-mon[30293]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:02:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:08 smithi107 ceph-mon[41097]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:02:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:08 smithi193 ceph-mon[33333]: pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:02:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:09.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:02:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:02:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:09 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]} : dispatch 2024-09-21T12:02:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:09 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]} : dispatch 2024-09-21T12:02:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:10.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:09 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]} : dispatch 2024-09-21T12:02:10.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:09 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]} : dispatch 2024-09-21T12:02:10.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:09 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]} : dispatch 2024-09-21T12:02:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:09 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]} : dispatch 2024-09-21T12:02:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:10 smithi193 ceph-mon[33333]: pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:02:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:10 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]': finished 2024-09-21T12:02:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:10 smithi193 ceph-mon[33333]: osdmap e407: 8 total, 8 up, 8 in 2024-09-21T12:02:11.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:10 smithi107 ceph-mon[30293]: pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:02:11.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:10 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]': finished 2024-09-21T12:02:11.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:10 smithi107 ceph-mon[30293]: osdmap e407: 8 total, 8 up, 8 in 2024-09-21T12:02:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:10 smithi107 ceph-mon[41097]: pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:02:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:10 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.17", "id": [5, 3]}]': finished 2024-09-21T12:02:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:10 smithi107 ceph-mon[41097]: osdmap e407: 8 total, 8 up, 8 in 2024-09-21T12:02:11.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:11 smithi193 ceph-mon[33333]: osdmap e408: 8 total, 8 up, 8 in 2024-09-21T12:02:12.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:11 smithi107 ceph-mon[30293]: osdmap e408: 8 total, 8 up, 8 in 2024-09-21T12:02:12.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:11 smithi107 ceph-mon[41097]: osdmap e408: 8 total, 8 up, 8 in 2024-09-21T12:02:12.543 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:02:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:02:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:12 smithi193 ceph-mon[33333]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:02:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-71"} : dispatch 2024-09-21T12:02:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-mon[30293]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:02:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-71"} : dispatch 2024-09-21T12:02:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-mon[41097]: pgmap v589: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:02:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-71"} : dispatch 2024-09-21T12:02:14.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:13 smithi107 ceph-mon[30293]: osdmap e409: 8 total, 8 up, 8 in 2024-09-21T12:02:14.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:13 smithi107 ceph-mon[30293]: osdmap e410: 8 total, 8 up, 8 in 2024-09-21T12:02:14.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:13 smithi107 ceph-mon[41097]: osdmap e409: 8 total, 8 up, 8 in 2024-09-21T12:02:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:13 smithi107 ceph-mon[41097]: osdmap e410: 8 total, 8 up, 8 in 2024-09-21T12:02:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:13 smithi193 ceph-mon[33333]: osdmap e409: 8 total, 8 up, 8 in 2024-09-21T12:02:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:13 smithi193 ceph-mon[33333]: osdmap e410: 8 total, 8 up, 8 in 2024-09-21T12:02:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-73","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[30293]: pgmap v591: 196 pgs: 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[30293]: osdmap e411: 8 total, 8 up, 8 in 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[41097]: pgmap v591: 196 pgs: 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[41097]: osdmap e411: 8 total, 8 up, 8 in 2024-09-21T12:02:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:14 smithi193 ceph-mon[33333]: pgmap v591: 196 pgs: 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:14 smithi193 ceph-mon[33333]: osdmap e411: 8 total, 8 up, 8 in 2024-09-21T12:02:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:16 smithi193 ceph-mon[33333]: pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:16.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:16 smithi193 ceph-mon[33333]: osdmap e412: 8 total, 8 up, 8 in 2024-09-21T12:02:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:16 smithi107 ceph-mon[30293]: pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:16 smithi107 ceph-mon[30293]: osdmap e412: 8 total, 8 up, 8 in 2024-09-21T12:02:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:16 smithi107 ceph-mon[41097]: pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:16 smithi107 ceph-mon[41097]: osdmap e412: 8 total, 8 up, 8 in 2024-09-21T12:02:18.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:17 smithi107 ceph-mon[30293]: osdmap e413: 8 total, 8 up, 8 in 2024-09-21T12:02:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:17 smithi107 ceph-mon[41097]: osdmap e413: 8 total, 8 up, 8 in 2024-09-21T12:02:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:17 smithi193 ceph-mon[33333]: osdmap e413: 8 total, 8 up, 8 in 2024-09-21T12:02:18.892 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:18 smithi193 ceph-mon[33333]: pgmap v597: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:18 smithi107 ceph-mon[30293]: pgmap v597: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:19.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:18 smithi107 ceph-mon[41097]: pgmap v597: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:19.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:02:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:02:20.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:20 smithi107 ceph-mon[30293]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:02:21.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:20 smithi107 ceph-mon[41097]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:02:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:20 smithi193 ceph-mon[33333]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:02:22.564 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:02:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:02:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:02:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:22 smithi107 ceph-mon[30293]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-21T12:02:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:22 smithi107 ceph-mon[41097]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-21T12:02:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:22 smithi193 ceph-mon[33333]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-21T12:02:24.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:02:24.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:02:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:02:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:02:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:02:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:23 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:02:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:25.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:24 smithi107 ceph-mon[41097]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-21T12:02:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:24 smithi107 ceph-mon[30293]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-21T12:02:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:24 smithi193 ceph-mon[33333]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-21T12:02:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:26 smithi107 ceph-mon[30293]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 725 B/s wr, 3 op/s 2024-09-21T12:02:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-73"} : dispatch 2024-09-21T12:02:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:26 smithi107 ceph-mon[41097]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 725 B/s wr, 3 op/s 2024-09-21T12:02:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-73"} : dispatch 2024-09-21T12:02:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:26 smithi193 ceph-mon[33333]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 725 B/s wr, 3 op/s 2024-09-21T12:02:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-73"} : dispatch 2024-09-21T12:02:28.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:27 smithi107 ceph-mon[30293]: osdmap e414: 8 total, 8 up, 8 in 2024-09-21T12:02:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:27 smithi107 ceph-mon[41097]: osdmap e414: 8 total, 8 up, 8 in 2024-09-21T12:02:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:27 smithi193 ceph-mon[33333]: osdmap e414: 8 total, 8 up, 8 in 2024-09-21T12:02:29.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:28 smithi107 ceph-mon[30293]: pgmap v603: 196 pgs: 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-09-21T12:02:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:28 smithi107 ceph-mon[30293]: osdmap e415: 8 total, 8 up, 8 in 2024-09-21T12:02:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:28 smithi107 ceph-mon[41097]: pgmap v603: 196 pgs: 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-09-21T12:02:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:28 smithi107 ceph-mon[41097]: osdmap e415: 8 total, 8 up, 8 in 2024-09-21T12:02:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:29.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:02:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:02:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:28 smithi193 ceph-mon[33333]: pgmap v603: 196 pgs: 196 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-09-21T12:02:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:28 smithi193 ceph-mon[33333]: osdmap e415: 8 total, 8 up, 8 in 2024-09-21T12:02:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-75","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:30.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:30.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[41097]: osdmap e416: 8 total, 8 up, 8 in 2024-09-21T12:02:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[30293]: osdmap e416: 8 total, 8 up, 8 in 2024-09-21T12:02:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:29 smithi193 ceph-mon[33333]: osdmap e416: 8 total, 8 up, 8 in 2024-09-21T12:02:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:30 smithi107 ceph-mon[30293]: pgmap v606: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:30 smithi107 ceph-mon[30293]: osdmap e417: 8 total, 8 up, 8 in 2024-09-21T12:02:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:30 smithi107 ceph-mon[41097]: pgmap v606: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:30 smithi107 ceph-mon[41097]: osdmap e417: 8 total, 8 up, 8 in 2024-09-21T12:02:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:30 smithi193 ceph-mon[33333]: pgmap v606: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:30 smithi193 ceph-mon[33333]: osdmap e417: 8 total, 8 up, 8 in 2024-09-21T12:02:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:31 smithi107 ceph-mon[30293]: osdmap e418: 8 total, 8 up, 8 in 2024-09-21T12:02:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-75"} : dispatch 2024-09-21T12:02:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:31 smithi107 ceph-mon[41097]: osdmap e418: 8 total, 8 up, 8 in 2024-09-21T12:02:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-75"} : dispatch 2024-09-21T12:02:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:31 smithi193 ceph-mon[33333]: osdmap e418: 8 total, 8 up, 8 in 2024-09-21T12:02:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-75"} : dispatch 2024-09-21T12:02:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:02:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:02:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:02:33.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:32 smithi107 ceph-mon[30293]: pgmap v609: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:02:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:32 smithi107 ceph-mon[30293]: osdmap e419: 8 total, 8 up, 8 in 2024-09-21T12:02:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:32 smithi107 ceph-mon[41097]: pgmap v609: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:02:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:32 smithi107 ceph-mon[41097]: osdmap e419: 8 total, 8 up, 8 in 2024-09-21T12:02:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:32 smithi193 ceph-mon[33333]: pgmap v609: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:02:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:32 smithi193 ceph-mon[33333]: osdmap e419: 8 total, 8 up, 8 in 2024-09-21T12:02:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:34 smithi193 ceph-mon[33333]: osdmap e420: 8 total, 8 up, 8 in 2024-09-21T12:02:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:34.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:34 smithi107 ceph-mon[41097]: osdmap e420: 8 total, 8 up, 8 in 2024-09-21T12:02:34.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:34 smithi107 ceph-mon[30293]: osdmap e420: 8 total, 8 up, 8 in 2024-09-21T12:02:34.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-77","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:35 smithi193 ceph-mon[33333]: pgmap v612: 228 pgs: 17 creating+peering, 15 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:02:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:35 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:02:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:35 smithi193 ceph-mon[33333]: osdmap e421: 8 total, 8 up, 8 in 2024-09-21T12:02:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:35.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[41097]: pgmap v612: 228 pgs: 17 creating+peering, 15 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:02:35.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[41097]: osdmap e421: 8 total, 8 up, 8 in 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[30293]: pgmap v612: 228 pgs: 17 creating+peering, 15 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[30293]: osdmap e421: 8 total, 8 up, 8 in 2024-09-21T12:02:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:36 smithi193 ceph-mon[33333]: osdmap e422: 8 total, 8 up, 8 in 2024-09-21T12:02:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-77"} : dispatch 2024-09-21T12:02:36.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:36 smithi107 ceph-mon[41097]: osdmap e422: 8 total, 8 up, 8 in 2024-09-21T12:02:36.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:36.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-77"} : dispatch 2024-09-21T12:02:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:36 smithi107 ceph-mon[30293]: osdmap e422: 8 total, 8 up, 8 in 2024-09-21T12:02:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:02:36.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-77"} : dispatch 2024-09-21T12:02:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:37 smithi193 ceph-mon[33333]: pgmap v615: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:02:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:37 smithi193 ceph-mon[33333]: osdmap e423: 8 total, 8 up, 8 in 2024-09-21T12:02:37.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:37 smithi107 ceph-mon[30293]: pgmap v615: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:02:37.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:37 smithi107 ceph-mon[30293]: osdmap e423: 8 total, 8 up, 8 in 2024-09-21T12:02:37.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:37 smithi107 ceph-mon[41097]: pgmap v615: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:02:37.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:37 smithi107 ceph-mon[41097]: osdmap e423: 8 total, 8 up, 8 in 2024-09-21T12:02:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:38 smithi193 ceph-mon[33333]: osdmap e424: 8 total, 8 up, 8 in 2024-09-21T12:02:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:38.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:38 smithi107 ceph-mon[30293]: osdmap e424: 8 total, 8 up, 8 in 2024-09-21T12:02:38.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:38 smithi107 ceph-mon[41097]: osdmap e424: 8 total, 8 up, 8 in 2024-09-21T12:02:38.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-79","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:02:39.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:02:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:02:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:39 smithi193 ceph-mon[33333]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:39 smithi193 ceph-mon[33333]: osdmap e425: 8 total, 8 up, 8 in 2024-09-21T12:02:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:39.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:39 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[41097]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[41097]: osdmap e425: 8 total, 8 up, 8 in 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[30293]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:02:39.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[30293]: osdmap e425: 8 total, 8 up, 8 in 2024-09-21T12:02:39.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:02:39.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:39 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:40.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:40 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:40.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:40 smithi107 ceph-mon[30293]: osdmap e426: 8 total, 8 up, 8 in 2024-09-21T12:02:40.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:40 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:40.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:40 smithi107 ceph-mon[41097]: osdmap e426: 8 total, 8 up, 8 in 2024-09-21T12:02:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:40 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:40.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:40 smithi193 ceph-mon[33333]: osdmap e426: 8 total, 8 up, 8 in 2024-09-21T12:02:41.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: osdmap e427: 8 total, 8 up, 8 in 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.1d"} : dispatch 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1d"}]: dispatch 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: 304.1d deep-scrub starts 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[30293]: 304.1d deep-scrub ok 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: osdmap e427: 8 total, 8 up, 8 in 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.1d"} : dispatch 2024-09-21T12:02:41.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1d"}]: dispatch 2024-09-21T12:02:41.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: 304.1d deep-scrub starts 2024-09-21T12:02:41.507 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:41 smithi107 ceph-mon[41097]: 304.1d deep-scrub ok 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: osdmap e427: 8 total, 8 up, 8 in 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "pg deep-scrub", "pgid": "304.1d"} : dispatch 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.1d"}]: dispatch 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: 304.1d deep-scrub starts 2024-09-21T12:02:41.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:41 smithi193 ceph-mon[33333]: 304.1d deep-scrub ok 2024-09-21T12:02:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:02:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:02:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:02:43.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:43 smithi107 ceph-mon[30293]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-21T12:02:43.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:43 smithi107 ceph-mon[41097]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-21T12:02:43.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:43 smithi193 ceph-mon[33333]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-21T12:02:45.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:45 smithi107 ceph-mon[41097]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:02:45.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:45 smithi107 ceph-mon[30293]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:02:45.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:45 smithi193 ceph-mon[33333]: pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T12:02:47.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:47 smithi107 ceph-mon[30293]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 969 B/s wr, 3 op/s 2024-09-21T12:02:47.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:47 smithi107 ceph-mon[41097]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 969 B/s wr, 3 op/s 2024-09-21T12:02:47.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:47 smithi193 ceph-mon[33333]: pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 969 B/s wr, 3 op/s 2024-09-21T12:02:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:49 smithi193 ceph-mon[33333]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 610 B/s rd, 244 B/s wr, 1 op/s 2024-09-21T12:02:49.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:02:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:02:49.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:49 smithi107 ceph-mon[30293]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 610 B/s rd, 244 B/s wr, 1 op/s 2024-09-21T12:02:49.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:49 smithi107 ceph-mon[41097]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 610 B/s rd, 244 B/s wr, 1 op/s 2024-09-21T12:02:50.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:50 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:50.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:50 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:50.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:50 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:02:51.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:51 smithi107 ceph-mon[41097]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-21T12:02:51.545 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:51 smithi107 ceph-mon[30293]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-21T12:02:51.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:51 smithi193 ceph-mon[33333]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-21T12:02:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:02:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:02:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:02:53.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:53 smithi107 ceph-mon[41097]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 899 B/s rd, 179 B/s wr, 1 op/s 2024-09-21T12:02:53.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:53 smithi107 ceph-mon[30293]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 899 B/s rd, 179 B/s wr, 1 op/s 2024-09-21T12:02:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:53 smithi193 ceph-mon[33333]: pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 899 B/s rd, 179 B/s wr, 1 op/s 2024-09-21T12:02:54.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:54 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:54.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:54 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:54 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:02:55.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:55 smithi107 ceph-mon[41097]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-21T12:02:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:55 smithi107 ceph-mon[30293]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-21T12:02:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:55 smithi193 ceph-mon[33333]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-21T12:02:57.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:57 smithi107 ceph-mon[41097]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:02:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:57 smithi107 ceph-mon[30293]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:02:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:57 smithi193 ceph-mon[33333]: pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:02:59.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:02:59 smithi193 ceph-mon[33333]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:02:59.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:02:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:02:59.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:02:59 smithi107 ceph-mon[41097]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:02:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:02:59 smithi107 ceph-mon[30293]: pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:03:00.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:00.504 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:03:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0[44809]: 2024-09-21T12:03:00.090+0000 7efeca538640 -1 snap_mapper.add_oid found existing snaps mapped on 304:bcb54a80:test-rados-api-smithi107-77755-80::foo:2, removing 2024-09-21T12:03:00.504 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:03:00 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T12:03:00.090+0000 7f3b645d8640 -1 snap_mapper.add_oid found existing snaps mapped on 304:bcb54a80:test-rados-api-smithi107-77755-80::foo:2, removing 2024-09-21T12:03:00.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:00.550 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:03:00 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:03:00.091+0000 7f58fda71640 -1 snap_mapper.add_oid found existing snaps mapped on 304:bcb54a80:test-rados-api-smithi107-77755-80::foo:2, removing 2024-09-21T12:03:01.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:01 smithi107 ceph-mon[41097]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-79"} : dispatch 2024-09-21T12:03:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:01 smithi107 ceph-mon[30293]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-79"} : dispatch 2024-09-21T12:03:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:01 smithi193 ceph-mon[33333]: pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-79"} : dispatch 2024-09-21T12:03:02.451 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:02 smithi193 ceph-mon[33333]: osdmap e428: 8 total, 8 up, 8 in 2024-09-21T12:03:02.451 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:02 smithi193 ceph-mon[33333]: osdmap e429: 8 total, 8 up, 8 in 2024-09-21T12:03:02.451 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:02.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:03:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:03:02.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-mon[30293]: osdmap e428: 8 total, 8 up, 8 in 2024-09-21T12:03:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-mon[30293]: osdmap e429: 8 total, 8 up, 8 in 2024-09-21T12:03:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-mon[41097]: osdmap e428: 8 total, 8 up, 8 in 2024-09-21T12:03:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-mon[41097]: osdmap e429: 8 total, 8 up, 8 in 2024-09-21T12:03:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-81","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:03.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[41097]: pgmap v634: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:03:03.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[41097]: osdmap e430: 8 total, 8 up, 8 in 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[30293]: pgmap v634: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[30293]: osdmap e430: 8 total, 8 up, 8 in 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:03.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:03.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:03 smithi193 ceph-mon[33333]: pgmap v634: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:03:03.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:03.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:03 smithi193 ceph-mon[33333]: osdmap e430: 8 total, 8 up, 8 in 2024-09-21T12:03:03.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:03.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:05.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[41097]: pgmap v637: 228 pgs: 18 unknown, 210 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[41097]: osdmap e431: 8 total, 8 up, 8 in 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[30293]: pgmap v637: 228 pgs: 18 unknown, 210 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[30293]: osdmap e431: 8 total, 8 up, 8 in 2024-09-21T12:03:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:05 smithi193 ceph-mon[33333]: pgmap v637: 228 pgs: 18 unknown, 210 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T12:03:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:05 smithi193 ceph-mon[33333]: osdmap e431: 8 total, 8 up, 8 in 2024-09-21T12:03:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:06.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:06.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:06 smithi107 ceph-mon[41097]: osdmap e432: 8 total, 8 up, 8 in 2024-09-21T12:03:06.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:06.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:06.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:06 smithi107 ceph-mon[30293]: osdmap e432: 8 total, 8 up, 8 in 2024-09-21T12:03:06.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:06.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:06.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:06 smithi193 ceph-mon[33333]: osdmap e432: 8 total, 8 up, 8 in 2024-09-21T12:03:06.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:07.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[41097]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[41097]: osdmap e433: 8 total, 8 up, 8 in 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[41097]: osdmap e434: 8 total, 8 up, 8 in 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[30293]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[30293]: osdmap e433: 8 total, 8 up, 8 in 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:07 smithi107 ceph-mon[30293]: osdmap e434: 8 total, 8 up, 8 in 2024-09-21T12:03:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:07 smithi193 ceph-mon[33333]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:03:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:07 smithi193 ceph-mon[33333]: osdmap e433: 8 total, 8 up, 8 in 2024-09-21T12:03:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:07 smithi193 ceph-mon[33333]: osdmap e434: 8 total, 8 up, 8 in 2024-09-21T12:03:08.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-21T12:03:08.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-21T12:03:08.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "512"} : dispatch 2024-09-21T12:03:09.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:03:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:03:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:09 smithi193 ceph-mon[33333]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:03:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-21T12:03:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:09 smithi193 ceph-mon[33333]: osdmap e435: 8 total, 8 up, 8 in 2024-09-21T12:03:09.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:09 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:09.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-21T12:03:09.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[30293]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:03:09.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-21T12:03:09.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[30293]: osdmap e435: 8 total, 8 up, 8 in 2024-09-21T12:03:09.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:09.405 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-21T12:03:09.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[41097]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:03:09.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-21T12:03:09.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[41097]: osdmap e435: 8 total, 8 up, 8 in 2024-09-21T12:03:09.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:09.406 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "16384"} : dispatch 2024-09-21T12:03:10.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[30293]: osdmap e436: 8 total, 8 up, 8 in 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[41097]: osdmap e436: 8 total, 8 up, 8 in 2024-09-21T12:03:10.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-21T12:03:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:10 smithi193 ceph-mon[33333]: osdmap e436: 8 total, 8 up, 8 in 2024-09-21T12:03:10.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:11.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[30293]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[30293]: osdmap e437: 8 total, 8 up, 8 in 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-81"} : dispatch 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[41097]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[41097]: osdmap e437: 8 total, 8 up, 8 in 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-81"} : dispatch 2024-09-21T12:03:11.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:11 smithi193 ceph-mon[33333]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-09-21T12:03:11.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:11.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:11 smithi193 ceph-mon[33333]: osdmap e437: 8 total, 8 up, 8 in 2024-09-21T12:03:11.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:11.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-81"} : dispatch 2024-09-21T12:03:12.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:12 smithi107 ceph-mon[30293]: osdmap e438: 8 total, 8 up, 8 in 2024-09-21T12:03:12.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:12 smithi107 ceph-mon[30293]: pgmap v649: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-09-21T12:03:12.504 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:03:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:03:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:03:12.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:12 smithi107 ceph-mon[41097]: osdmap e438: 8 total, 8 up, 8 in 2024-09-21T12:03:12.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:12 smithi107 ceph-mon[41097]: pgmap v649: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-09-21T12:03:12.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:12 smithi193 ceph-mon[33333]: osdmap e438: 8 total, 8 up, 8 in 2024-09-21T12:03:12.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:12 smithi193 ceph-mon[33333]: pgmap v649: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-09-21T12:03:13.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:13 smithi107 ceph-mon[30293]: osdmap e439: 8 total, 8 up, 8 in 2024-09-21T12:03:13.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:13.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:13 smithi107 ceph-mon[41097]: osdmap e439: 8 total, 8 up, 8 in 2024-09-21T12:03:13.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:13.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:13 smithi193 ceph-mon[33333]: osdmap e439: 8 total, 8 up, 8 in 2024-09-21T12:03:13.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-83","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: osdmap e440: 8 total, 8 up, 8 in 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: pgmap v652: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: osdmap e441: 8 total, 8 up, 8 in 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:14.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: osdmap e440: 8 total, 8 up, 8 in 2024-09-21T12:03:14.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:14.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:14.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: pgmap v652: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T12:03:14.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:14.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: osdmap e441: 8 total, 8 up, 8 in 2024-09-21T12:03:14.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:14.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:14.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: osdmap e440: 8 total, 8 up, 8 in 2024-09-21T12:03:14.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:14.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:14.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: pgmap v652: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T12:03:14.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:14.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: osdmap e441: 8 total, 8 up, 8 in 2024-09-21T12:03:14.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:16.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[30293]: osdmap e442: 8 total, 8 up, 8 in 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[30293]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[41097]: osdmap e442: 8 total, 8 up, 8 in 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:16.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:16 smithi107 ceph-mon[41097]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-09-21T12:03:16.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:16.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:16 smithi193 ceph-mon[33333]: osdmap e442: 8 total, 8 up, 8 in 2024-09-21T12:03:16.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:16.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:16 smithi193 ceph-mon[33333]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-09-21T12:03:17.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:17.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:17 smithi107 ceph-mon[30293]: osdmap e443: 8 total, 8 up, 8 in 2024-09-21T12:03:17.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:17.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:17.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:17 smithi107 ceph-mon[41097]: osdmap e443: 8 total, 8 up, 8 in 2024-09-21T12:03:17.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:17 smithi193 ceph-mon[33333]: osdmap e443: 8 total, 8 up, 8 in 2024-09-21T12:03:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:18.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:18.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:18 smithi107 ceph-mon[30293]: osdmap e444: 8 total, 8 up, 8 in 2024-09-21T12:03:18.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:18 smithi107 ceph-mon[30293]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T12:03:18.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:18.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:18 smithi107 ceph-mon[41097]: osdmap e444: 8 total, 8 up, 8 in 2024-09-21T12:03:18.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:18 smithi107 ceph-mon[41097]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T12:03:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:18 smithi193 ceph-mon[33333]: osdmap e444: 8 total, 8 up, 8 in 2024-09-21T12:03:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:18 smithi193 ceph-mon[33333]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-21T12:03:19.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:03:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:03:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:19 smithi193 ceph-mon[33333]: osdmap e445: 8 total, 8 up, 8 in 2024-09-21T12:03:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:19.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:19 smithi107 ceph-mon[30293]: osdmap e445: 8 total, 8 up, 8 in 2024-09-21T12:03:19.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:19.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:19 smithi107 ceph-mon[41097]: osdmap e445: 8 total, 8 up, 8 in 2024-09-21T12:03:19.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:20.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[30293]: osdmap e446: 8 total, 8 up, 8 in 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-83"} : dispatch 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[30293]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[41097]: osdmap e446: 8 total, 8 up, 8 in 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-83"} : dispatch 2024-09-21T12:03:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:20 smithi107 ceph-mon[41097]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-21T12:03:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:20 smithi193 ceph-mon[33333]: osdmap e446: 8 total, 8 up, 8 in 2024-09-21T12:03:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-83"} : dispatch 2024-09-21T12:03:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:20 smithi193 ceph-mon[33333]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-21T12:03:21.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:21 smithi107 ceph-mon[30293]: osdmap e447: 8 total, 8 up, 8 in 2024-09-21T12:03:21.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:21 smithi107 ceph-mon[41097]: osdmap e447: 8 total, 8 up, 8 in 2024-09-21T12:03:21.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:21 smithi193 ceph-mon[33333]: osdmap e447: 8 total, 8 up, 8 in 2024-09-21T12:03:22.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:03:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:03:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-mon[30293]: osdmap e448: 8 total, 8 up, 8 in 2024-09-21T12:03:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-mon[30293]: pgmap v664: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:22.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-mon[41097]: osdmap e448: 8 total, 8 up, 8 in 2024-09-21T12:03:22.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:22.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:22 smithi107 ceph-mon[41097]: pgmap v664: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:22.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:22 smithi193 ceph-mon[33333]: osdmap e448: 8 total, 8 up, 8 in 2024-09-21T12:03:22.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-85","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:22.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:22 smithi193 ceph-mon[33333]: pgmap v664: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:23.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:23.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:23.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: osdmap e449: 8 total, 8 up, 8 in 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: osdmap e449: 8 total, 8 up, 8 in 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:03:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: osdmap e449: 8 total, 8 up, 8 in 2024-09-21T12:03:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:23.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:23.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: osdmap e450: 8 total, 8 up, 8 in 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: pgmap v667: 228 pgs: 14 unknown, 214 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: osdmap e450: 8 total, 8 up, 8 in 2024-09-21T12:03:24.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: pgmap v667: 228 pgs: 14 unknown, 214 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:03:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:03:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:03:24.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:03:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: osdmap e450: 8 total, 8 up, 8 in 2024-09-21T12:03:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: pgmap v667: 228 pgs: 14 unknown, 214 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:03:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:03:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:03:24.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:03:25.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:25.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:25 smithi107 ceph-mon[30293]: osdmap e451: 8 total, 8 up, 8 in 2024-09-21T12:03:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:25 smithi107 ceph-mon[41097]: osdmap e451: 8 total, 8 up, 8 in 2024-09-21T12:03:25.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_tier","val": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:03:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:25 smithi193 ceph-mon[33333]: osdmap e451: 8 total, 8 up, 8 in 2024-09-21T12:03:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[30293]: osdmap e452: 8 total, 8 up, 8 in 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[30293]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[30293]: osdmap e453: 8 total, 8 up, 8 in 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[41097]: osdmap e452: 8 total, 8 up, 8 in 2024-09-21T12:03:26.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:26.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[41097]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-21T12:03:26.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:26.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:26 smithi107 ceph-mon[41097]: osdmap e453: 8 total, 8 up, 8 in 2024-09-21T12:03:26.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:26.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:26 smithi193 ceph-mon[33333]: osdmap e452: 8 total, 8 up, 8 in 2024-09-21T12:03:26.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:26.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:26 smithi193 ceph-mon[33333]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-21T12:03:26.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:26.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:26 smithi193 ceph-mon[33333]: osdmap e453: 8 total, 8 up, 8 in 2024-09-21T12:03:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:28 smithi193 ceph-mon[33333]: osdmap e454: 8 total, 8 up, 8 in 2024-09-21T12:03:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:28 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:28.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:28 smithi193 ceph-mon[33333]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-21T12:03:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:28 smithi107 ceph-mon[30293]: osdmap e454: 8 total, 8 up, 8 in 2024-09-21T12:03:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:28 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:28 smithi107 ceph-mon[30293]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-21T12:03:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:28 smithi107 ceph-mon[41097]: osdmap e454: 8 total, 8 up, 8 in 2024-09-21T12:03:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:28 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:28 smithi107 ceph-mon[41097]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-21T12:03:29.279 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:03:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:03:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:29 smithi193 ceph-mon[33333]: osdmap e455: 8 total, 8 up, 8 in 2024-09-21T12:03:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-85"} : dispatch 2024-09-21T12:03:29.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:29.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[30293]: osdmap e455: 8 total, 8 up, 8 in 2024-09-21T12:03:29.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:29.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-85"} : dispatch 2024-09-21T12:03:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[41097]: osdmap e455: 8 total, 8 up, 8 in 2024-09-21T12:03:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-85"} : dispatch 2024-09-21T12:03:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:30 smithi193 ceph-mon[33333]: osdmap e456: 8 total, 8 up, 8 in 2024-09-21T12:03:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:30 smithi193 ceph-mon[33333]: pgmap v676: 196 pgs: 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T12:03:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:30 smithi107 ceph-mon[30293]: osdmap e456: 8 total, 8 up, 8 in 2024-09-21T12:03:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:30 smithi107 ceph-mon[30293]: pgmap v676: 196 pgs: 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T12:03:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:30 smithi107 ceph-mon[41097]: osdmap e456: 8 total, 8 up, 8 in 2024-09-21T12:03:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:30 smithi107 ceph-mon[41097]: pgmap v676: 196 pgs: 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T12:03:31.298 INFO:tasks.workunit.client.0.smithi107.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18095 ms) 2024-09-21T12:03:31.298 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22360 ms) 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3017 ms) 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3031 ms) 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3017 ms) 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3027 ms) 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.299 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4031 ms) 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38379 ms) 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16825 ms) 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4015 ms) 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3010 ms) 2024-09-21T12:03:31.300 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24154 ms) 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14427 ms) 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5036 ms) 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4030 ms) 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.301 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: waiting for scrubs... 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: done waiting 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25073 ms) 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10057 ms) 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9046 ms) 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9072 ms) 2024-09-21T12:03:31.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-21T12:03:31.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:31 smithi107 ceph-mon[30293]: osdmap e457: 8 total, 8 up, 8 in 2024-09-21T12:03:31.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:31.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:31 smithi107 ceph-mon[41097]: osdmap e457: 8 total, 8 up, 8 in 2024-09-21T12:03:31.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:31 smithi193 ceph-mon[33333]: osdmap e457: 8 total, 8 up, 8 in 2024-09-21T12:03:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-87","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:32.702 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:32.702 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:32 smithi193 ceph-mon[33333]: osdmap e458: 8 total, 8 up, 8 in 2024-09-21T12:03:32.702 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:32.702 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:32 smithi193 ceph-mon[33333]: pgmap v679: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T12:03:32.755 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:03:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:03:32.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:32.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[30293]: osdmap e458: 8 total, 8 up, 8 in 2024-09-21T12:03:32.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:32.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[30293]: pgmap v679: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T12:03:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:32.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[41097]: osdmap e458: 8 total, 8 up, 8 in 2024-09-21T12:03:32.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:32.756 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:32 smithi107 ceph-mon[41097]: pgmap v679: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-21T12:03:33.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:33 smithi107 ceph-mon[30293]: osdmap e459: 8 total, 8 up, 8 in 2024-09-21T12:03:33.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:33 smithi107 ceph-mon[41097]: osdmap e459: 8 total, 8 up, 8 in 2024-09-21T12:03:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:33 smithi193 ceph-mon[33333]: osdmap e459: 8 total, 8 up, 8 in 2024-09-21T12:03:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:34.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[30293]: osdmap e460: 8 total, 8 up, 8 in 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_tier","val": "test-rados-api-smithi107-77755-89-test-flush"} : dispatch 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[30293]: pgmap v682: 260 pgs: 26 unknown, 234 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[41097]: osdmap e460: 8 total, 8 up, 8 in 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_tier","val": "test-rados-api-smithi107-77755-89-test-flush"} : dispatch 2024-09-21T12:03:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:34 smithi107 ceph-mon[41097]: pgmap v682: 260 pgs: 26 unknown, 234 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:34 smithi193 ceph-mon[33333]: osdmap e460: 8 total, 8 up, 8 in 2024-09-21T12:03:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_tier","val": "test-rados-api-smithi107-77755-89-test-flush"} : dispatch 2024-09-21T12:03:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:34 smithi193 ceph-mon[33333]: pgmap v682: 260 pgs: 26 unknown, 234 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:35.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_tier","val": "test-rados-api-smithi107-77755-89-test-flush"}]': finished 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[30293]: osdmap e461: 8 total, 8 up, 8 in 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_tier","val": "test-rados-api-smithi107-77755-89-test-flush"}]': finished 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[41097]: osdmap e461: 8 total, 8 up, 8 in 2024-09-21T12:03:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:35 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_tier","val": "test-rados-api-smithi107-77755-89-test-flush"}]': finished 2024-09-21T12:03:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:35 smithi193 ceph-mon[33333]: osdmap e461: 8 total, 8 up, 8 in 2024-09-21T12:03:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:03:36.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[30293]: osdmap e462: 8 total, 8 up, 8 in 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[30293]: pgmap v685: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[41097]: osdmap e462: 8 total, 8 up, 8 in 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[41097]: pgmap v685: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:03:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:36 smithi193 ceph-mon[33333]: osdmap e462: 8 total, 8 up, 8 in 2024-09-21T12:03:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:03:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:36 smithi193 ceph-mon[33333]: pgmap v685: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:03:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:03:37.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:37 smithi107 ceph-mon[30293]: osdmap e463: 8 total, 8 up, 8 in 2024-09-21T12:03:37.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:37 smithi107 ceph-mon[41097]: osdmap e463: 8 total, 8 up, 8 in 2024-09-21T12:03:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:37 smithi193 ceph-mon[33333]: osdmap e463: 8 total, 8 up, 8 in 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[30293]: osdmap e464: 8 total, 8 up, 8 in 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-87"} : dispatch 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[30293]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[41097]: osdmap e464: 8 total, 8 up, 8 in 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-87"} : dispatch 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[41097]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:03:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:38 smithi193 ceph-mon[33333]: osdmap e464: 8 total, 8 up, 8 in 2024-09-21T12:03:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:38.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-87"} : dispatch 2024-09-21T12:03:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:38 smithi193 ceph-mon[33333]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:03:38.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:39.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:03:38 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:03:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:39 smithi107 ceph-mon[41097]: osdmap e465: 8 total, 8 up, 8 in 2024-09-21T12:03:39.655 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:39.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:39 smithi107 ceph-mon[30293]: osdmap e465: 8 total, 8 up, 8 in 2024-09-21T12:03:39.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:39 smithi193 ceph-mon[33333]: osdmap e465: 8 total, 8 up, 8 in 2024-09-21T12:03:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:40.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:40 smithi107 ceph-mon[41097]: osdmap e466: 8 total, 8 up, 8 in 2024-09-21T12:03:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:40 smithi107 ceph-mon[41097]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:40 smithi107 ceph-mon[30293]: osdmap e466: 8 total, 8 up, 8 in 2024-09-21T12:03:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:40 smithi107 ceph-mon[30293]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:40 smithi193 ceph-mon[33333]: osdmap e466: 8 total, 8 up, 8 in 2024-09-21T12:03:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-90","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:40 smithi193 ceph-mon[33333]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:41.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[41097]: osdmap e467: 8 total, 8 up, 8 in 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[30293]: osdmap e467: 8 total, 8 up, 8 in 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:41 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:41 smithi193 ceph-mon[33333]: osdmap e467: 8 total, 8 up, 8 in 2024-09-21T12:03:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:03:42.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-mon[30293]: osdmap e468: 8 total, 8 up, 8 in 2024-09-21T12:03:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-mon[30293]: pgmap v694: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:42.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:03:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:03:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-mon[41097]: osdmap e468: 8 total, 8 up, 8 in 2024-09-21T12:03:42.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:42 smithi107 ceph-mon[41097]: pgmap v694: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:03:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:42 smithi193 ceph-mon[33333]: osdmap e468: 8 total, 8 up, 8 in 2024-09-21T12:03:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:42 smithi193 ceph-mon[33333]: pgmap v694: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:03:43.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:43 smithi107 ceph-mon[41097]: osdmap e469: 8 total, 8 up, 8 in 2024-09-21T12:03:43.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:43 smithi107 ceph-mon[30293]: osdmap e469: 8 total, 8 up, 8 in 2024-09-21T12:03:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:43 smithi193 ceph-mon[33333]: osdmap e469: 8 total, 8 up, 8 in 2024-09-21T12:03:44.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[30293]: osdmap e470: 8 total, 8 up, 8 in 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-90"} : dispatch 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[30293]: pgmap v697: 228 pgs: 15 unknown, 213 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[41097]: osdmap e470: 8 total, 8 up, 8 in 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-90"} : dispatch 2024-09-21T12:03:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:44 smithi107 ceph-mon[41097]: pgmap v697: 228 pgs: 15 unknown, 213 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:03:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:44 smithi193 ceph-mon[33333]: osdmap e470: 8 total, 8 up, 8 in 2024-09-21T12:03:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-90"} : dispatch 2024-09-21T12:03:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:44 smithi193 ceph-mon[33333]: pgmap v697: 228 pgs: 15 unknown, 213 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:03:45.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:45 smithi107 ceph-mon[41097]: osdmap e471: 8 total, 8 up, 8 in 2024-09-21T12:03:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:45 smithi107 ceph-mon[41097]: osdmap e472: 8 total, 8 up, 8 in 2024-09-21T12:03:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:45 smithi107 ceph-mon[30293]: osdmap e471: 8 total, 8 up, 8 in 2024-09-21T12:03:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:45 smithi107 ceph-mon[30293]: osdmap e472: 8 total, 8 up, 8 in 2024-09-21T12:03:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:45 smithi193 ceph-mon[33333]: osdmap e471: 8 total, 8 up, 8 in 2024-09-21T12:03:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:45 smithi193 ceph-mon[33333]: osdmap e472: 8 total, 8 up, 8 in 2024-09-21T12:03:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-92","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:46.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:46 smithi107 ceph-mon[30293]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T12:03:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:46 smithi107 ceph-mon[30293]: osdmap e473: 8 total, 8 up, 8 in 2024-09-21T12:03:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:46 smithi107 ceph-mon[41097]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T12:03:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:46.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:46 smithi107 ceph-mon[41097]: osdmap e473: 8 total, 8 up, 8 in 2024-09-21T12:03:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:46 smithi193 ceph-mon[33333]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T12:03:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:46 smithi193 ceph-mon[33333]: osdmap e473: 8 total, 8 up, 8 in 2024-09-21T12:03:47.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:47.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:47 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:47 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:47 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:48.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:48 smithi107 ceph-mon[30293]: osdmap e474: 8 total, 8 up, 8 in 2024-09-21T12:03:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:48 smithi107 ceph-mon[30293]: pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T12:03:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:48 smithi107 ceph-mon[41097]: osdmap e474: 8 total, 8 up, 8 in 2024-09-21T12:03:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:48 smithi107 ceph-mon[41097]: pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T12:03:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:48 smithi193 ceph-mon[33333]: osdmap e474: 8 total, 8 up, 8 in 2024-09-21T12:03:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:48 smithi193 ceph-mon[33333]: pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-21T12:03:49.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:03:48 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:03:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[30293]: osdmap e475: 8 total, 8 up, 8 in 2024-09-21T12:03:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-92"} : dispatch 2024-09-21T12:03:49.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[41097]: osdmap e475: 8 total, 8 up, 8 in 2024-09-21T12:03:49.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-92"} : dispatch 2024-09-21T12:03:49.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:49 smithi193 ceph-mon[33333]: osdmap e475: 8 total, 8 up, 8 in 2024-09-21T12:03:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:03:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-92"} : dispatch 2024-09-21T12:03:49.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:03:50.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:50 smithi107 ceph-mon[41097]: pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 1.2 KiB/s wr, 179 op/s 2024-09-21T12:03:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:50 smithi107 ceph-mon[41097]: osdmap e476: 8 total, 8 up, 8 in 2024-09-21T12:03:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:50 smithi107 ceph-mon[30293]: pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 1.2 KiB/s wr, 179 op/s 2024-09-21T12:03:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:50 smithi107 ceph-mon[30293]: osdmap e476: 8 total, 8 up, 8 in 2024-09-21T12:03:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:50 smithi193 ceph-mon[33333]: pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 1.2 KiB/s wr, 179 op/s 2024-09-21T12:03:50.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:50 smithi193 ceph-mon[33333]: osdmap e476: 8 total, 8 up, 8 in 2024-09-21T12:03:51.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:51 smithi107 ceph-mon[41097]: osdmap e477: 8 total, 8 up, 8 in 2024-09-21T12:03:51.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:51 smithi107 ceph-mon[30293]: osdmap e477: 8 total, 8 up, 8 in 2024-09-21T12:03:51.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:51 smithi193 ceph-mon[33333]: osdmap e477: 8 total, 8 up, 8 in 2024-09-21T12:03:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-94","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:03:52.754 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:03:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:03:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[41097]: pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 509 B/s wr, 179 op/s 2024-09-21T12:03:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[41097]: osdmap e478: 8 total, 8 up, 8 in 2024-09-21T12:03:52.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[30293]: pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 509 B/s wr, 179 op/s 2024-09-21T12:03:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[30293]: osdmap e478: 8 total, 8 up, 8 in 2024-09-21T12:03:52.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:52 smithi193 ceph-mon[33333]: pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 509 B/s wr, 179 op/s 2024-09-21T12:03:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:52 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:03:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:52 smithi193 ceph-mon[33333]: osdmap e478: 8 total, 8 up, 8 in 2024-09-21T12:03:52.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:03:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:53 smithi107 ceph-mon[30293]: osdmap e479: 8 total, 8 up, 8 in 2024-09-21T12:03:53.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:53 smithi107 ceph-mon[41097]: osdmap e479: 8 total, 8 up, 8 in 2024-09-21T12:03:53.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:53 smithi193 ceph-mon[33333]: osdmap e479: 8 total, 8 up, 8 in 2024-09-21T12:03:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:03:54.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:54 smithi107 ceph-mon[30293]: pgmap v711: 228 pgs: 12 unknown, 216 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:03:54.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:54 smithi107 ceph-mon[30293]: osdmap e480: 8 total, 8 up, 8 in 2024-09-21T12:03:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:54 smithi107 ceph-mon[41097]: pgmap v711: 228 pgs: 12 unknown, 216 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:03:54.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:54 smithi107 ceph-mon[41097]: osdmap e480: 8 total, 8 up, 8 in 2024-09-21T12:03:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:54 smithi193 ceph-mon[33333]: pgmap v711: 228 pgs: 12 unknown, 216 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:03:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:54 smithi193 ceph-mon[33333]: osdmap e480: 8 total, 8 up, 8 in 2024-09-21T12:03:56.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:56 smithi107 ceph-mon[41097]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-21T12:03:56.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:56 smithi107 ceph-mon[30293]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-21T12:03:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:56 smithi193 ceph-mon[33333]: pgmap v713: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-09-21T12:03:57.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:57 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:57.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:57 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:57 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:03:58.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:58 smithi107 ceph-mon[41097]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T12:03:58.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:58 smithi107 ceph-mon[30293]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T12:03:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:58 smithi193 ceph-mon[33333]: pgmap v714: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-21T12:03:59.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:03:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:03:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:03:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:00.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:03:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:03:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:00 smithi193 ceph-mon[33333]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-21T12:04:01.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:00 smithi107 ceph-mon[30293]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-21T12:04:01.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:00 smithi107 ceph-mon[41097]: pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-21T12:04:02.508 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:04:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:04:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:04:02.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:02 smithi193 ceph-mon[33333]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-21T12:04:03.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:02 smithi107 ceph-mon[30293]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-21T12:04:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:02 smithi107 ceph-mon[41097]: pgmap v716: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-21T12:04:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:04 smithi193 ceph-mon[33333]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-21T12:04:05.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:04 smithi107 ceph-mon[30293]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-21T12:04:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:04 smithi107 ceph-mon[41097]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 409 B/s wr, 3 op/s 2024-09-21T12:04:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:05 smithi193 ceph-mon[33333]: osdmap e481: 8 total, 8 up, 8 in 2024-09-21T12:04:06.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:05 smithi107 ceph-mon[30293]: osdmap e481: 8 total, 8 up, 8 in 2024-09-21T12:04:06.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:05 smithi107 ceph-mon[41097]: osdmap e481: 8 total, 8 up, 8 in 2024-09-21T12:04:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:06 smithi193 ceph-mon[33333]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:07.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:06 smithi107 ceph-mon[41097]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:07.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:06 smithi107 ceph-mon[30293]: pgmap v719: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:08 smithi193 ceph-mon[33333]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:09.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:08 smithi107 ceph-mon[41097]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:09.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:08 smithi107 ceph-mon[30293]: pgmap v720: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:09.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:04:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:04:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:09.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:09.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:10 smithi193 ceph-mon[33333]: pgmap v721: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:11.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:10 smithi107 ceph-mon[41097]: pgmap v721: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:11.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:10 smithi107 ceph-mon[30293]: pgmap v721: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:12.547 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:04:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:04:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:04:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:12 smithi193 ceph-mon[33333]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:12.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:12 smithi193 ceph-mon[33333]: osdmap e482: 8 total, 8 up, 8 in 2024-09-21T12:04:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:12 smithi107 ceph-mon[30293]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:12 smithi107 ceph-mon[30293]: osdmap e482: 8 total, 8 up, 8 in 2024-09-21T12:04:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:12 smithi107 ceph-mon[41097]: pgmap v722: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:12 smithi107 ceph-mon[41097]: osdmap e482: 8 total, 8 up, 8 in 2024-09-21T12:04:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:14 smithi107 ceph-mon[30293]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-94"} : dispatch 2024-09-21T12:04:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:14 smithi107 ceph-mon[41097]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-94"} : dispatch 2024-09-21T12:04:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:14 smithi193 ceph-mon[33333]: pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-94"} : dispatch 2024-09-21T12:04:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:16 smithi107 ceph-mon[30293]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:16 smithi107 ceph-mon[30293]: osdmap e483: 8 total, 8 up, 8 in 2024-09-21T12:04:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:16 smithi107 ceph-mon[41097]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:16 smithi107 ceph-mon[41097]: osdmap e483: 8 total, 8 up, 8 in 2024-09-21T12:04:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:16 smithi193 ceph-mon[33333]: pgmap v725: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:16 smithi193 ceph-mon[33333]: osdmap e483: 8 total, 8 up, 8 in 2024-09-21T12:04:18.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:17 smithi107 ceph-mon[30293]: osdmap e484: 8 total, 8 up, 8 in 2024-09-21T12:04:18.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:17 smithi107 ceph-mon[41097]: osdmap e484: 8 total, 8 up, 8 in 2024-09-21T12:04:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:17 smithi193 ceph-mon[33333]: osdmap e484: 8 total, 8 up, 8 in 2024-09-21T12:04:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-96","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[30293]: pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[30293]: osdmap e485: 8 total, 8 up, 8 in 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[41097]: pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[41097]: osdmap e485: 8 total, 8 up, 8 in 2024-09-21T12:04:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:04:19.012 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:18 smithi193 ceph-mon[33333]: pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:19.012 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:18 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:19.012 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:19.012 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:18 smithi193 ceph-mon[33333]: osdmap e485: 8 total, 8 up, 8 in 2024-09-21T12:04:19.012 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:04:19.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:04:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:04:20.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:19 smithi107 ceph-mon[41097]: osdmap e486: 8 total, 8 up, 8 in 2024-09-21T12:04:20.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:19 smithi107 ceph-mon[30293]: osdmap e486: 8 total, 8 up, 8 in 2024-09-21T12:04:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:19 smithi193 ceph-mon[33333]: osdmap e486: 8 total, 8 up, 8 in 2024-09-21T12:04:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:21.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:20 smithi107 ceph-mon[41097]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:20 smithi107 ceph-mon[30293]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:20 smithi193 ceph-mon[33333]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:22.588 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:04:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:04:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:04:23.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:22 smithi107 ceph-mon[30293]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 867 B/s rd, 867 B/s wr, 1 op/s 2024-09-21T12:04:23.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:22 smithi107 ceph-mon[41097]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 867 B/s rd, 867 B/s wr, 1 op/s 2024-09-21T12:04:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:22 smithi193 ceph-mon[33333]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 867 B/s rd, 867 B/s wr, 1 op/s 2024-09-21T12:04:24.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:24.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 891 B/s wr, 2 op/s 2024-09-21T12:04:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:04:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 891 B/s wr, 2 op/s 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.007 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:04:25.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:04:25.008 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 891 B/s wr, 2 op/s 2024-09-21T12:04:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:04:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T12:04:25.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:25.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:04:25.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:04:25.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:04:26.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:25 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi107 to 3247M 2024-09-21T12:04:26.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:25 smithi107 ceph-mon[30293]: Adjusting osd_memory_target on smithi193 to 3248M 2024-09-21T12:04:26.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:25 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi107 to 3247M 2024-09-21T12:04:26.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:25 smithi107 ceph-mon[41097]: Adjusting osd_memory_target on smithi193 to 3248M 2024-09-21T12:04:26.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:25 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi107 to 3247M 2024-09-21T12:04:26.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:25 smithi193 ceph-mon[33333]: Adjusting osd_memory_target on smithi193 to 3248M 2024-09-21T12:04:27.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:26 smithi107 ceph-mon[30293]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:04:27.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:26 smithi107 ceph-mon[41097]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:04:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:26 smithi193 ceph-mon[33333]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-21T12:04:28.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:27 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:27 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:27 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:29.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:28 smithi107 ceph-mon[41097]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 621 B/s wr, 2 op/s 2024-09-21T12:04:29.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:28 smithi107 ceph-mon[30293]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 621 B/s wr, 2 op/s 2024-09-21T12:04:29.004 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:04:28 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T12:04:28.602+0000 7f3b695e2640 -1 snap_mapper.add_oid found existing snaps mapped on 109:cb1996f0:test-rados-api-smithi107-77755-97::foo:21, removing 2024-09-21T12:04:29.018 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:28 smithi193 ceph-mon[33333]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 621 B/s wr, 2 op/s 2024-09-21T12:04:29.018 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:04:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:04:28.602+0000 7f58fda71640 -1 snap_mapper.add_oid found existing snaps mapped on 109:cb1996f0:test-rados-api-smithi107-77755-97::foo:21, removing 2024-09-21T12:04:29.019 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:04:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:04:28.602+0000 7f7a4241f640 -1 snap_mapper.add_oid found existing snaps mapped on 109:cb1996f0:test-rados-api-smithi107-77755-97::foo:21, removing 2024-09-21T12:04:29.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:04:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:04:30.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:30.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-96"} : dispatch 2024-09-21T12:04:30.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:30.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:30.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-96"} : dispatch 2024-09-21T12:04:30.006 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-96"} : dispatch 2024-09-21T12:04:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:30 smithi107 ceph-mon[30293]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 94 B/s wr, 1 op/s 2024-09-21T12:04:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:30 smithi107 ceph-mon[30293]: osdmap e487: 8 total, 8 up, 8 in 2024-09-21T12:04:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:30 smithi107 ceph-mon[41097]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 94 B/s wr, 1 op/s 2024-09-21T12:04:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:30 smithi107 ceph-mon[41097]: osdmap e487: 8 total, 8 up, 8 in 2024-09-21T12:04:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:30 smithi193 ceph-mon[33333]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 94 B/s wr, 1 op/s 2024-09-21T12:04:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:30 smithi193 ceph-mon[33333]: osdmap e487: 8 total, 8 up, 8 in 2024-09-21T12:04:32.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:31 smithi107 ceph-mon[30293]: osdmap e488: 8 total, 8 up, 8 in 2024-09-21T12:04:32.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:31 smithi107 ceph-mon[41097]: osdmap e488: 8 total, 8 up, 8 in 2024-09-21T12:04:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:31 smithi193 ceph-mon[33333]: osdmap e488: 8 total, 8 up, 8 in 2024-09-21T12:04:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-98","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:32.641 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:04:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:04:32.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:32 smithi193 ceph-mon[33333]: pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:04:32.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:32.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:32 smithi193 ceph-mon[33333]: osdmap e489: 8 total, 8 up, 8 in 2024-09-21T12:04:32.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:04:32.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:32 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:33.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[30293]: pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:04:33.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[30293]: osdmap e489: 8 total, 8 up, 8 in 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[41097]: pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[41097]: osdmap e489: 8 total, 8 up, 8 in 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:04:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:32 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:34.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:04:34.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:33 smithi107 ceph-mon[30293]: osdmap e490: 8 total, 8 up, 8 in 2024-09-21T12:04:34.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-98", "mode": "writeback"} : dispatch 2024-09-21T12:04:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:04:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:33 smithi107 ceph-mon[41097]: osdmap e490: 8 total, 8 up, 8 in 2024-09-21T12:04:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-98", "mode": "writeback"} : dispatch 2024-09-21T12:04:34.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:04:34.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:33 smithi193 ceph-mon[33333]: osdmap e490: 8 total, 8 up, 8 in 2024-09-21T12:04:34.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-98", "mode": "writeback"} : dispatch 2024-09-21T12:04:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[30293]: pgmap v742: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-21T12:04:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:04:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-98", "mode": "writeback"}]': finished 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[30293]: osdmap e491: 8 total, 8 up, 8 in 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-98"} : dispatch 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[41097]: pgmap v742: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-98", "mode": "writeback"}]': finished 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[41097]: osdmap e491: 8 total, 8 up, 8 in 2024-09-21T12:04:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-98"} : dispatch 2024-09-21T12:04:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:34 smithi193 ceph-mon[33333]: pgmap v742: 228 pgs: 13 unknown, 215 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-21T12:04:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:34 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:04:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-98", "mode": "writeback"}]': finished 2024-09-21T12:04:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:34 smithi193 ceph-mon[33333]: osdmap e491: 8 total, 8 up, 8 in 2024-09-21T12:04:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-98"} : dispatch 2024-09-21T12:04:36.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-98"}]': finished 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: osdmap e492: 8 total, 8 up, 8 in 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: osdmap e493: 8 total, 8 up, 8 in 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-98"}]': finished 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: osdmap e492: 8 total, 8 up, 8 in 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:04:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:04:36.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:04:36.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: osdmap e493: 8 total, 8 up, 8 in 2024-09-21T12:04:36.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-98"}]': finished 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: osdmap e492: 8 total, 8 up, 8 in 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: osdmap e493: 8 total, 8 up, 8 in 2024-09-21T12:04:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_count","val": "1"} : dispatch 2024-09-21T12:04:37.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[30293]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:04:37.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_count","val": "1"}]': finished 2024-09-21T12:04:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[30293]: osdmap e494: 8 total, 8 up, 8 in 2024-09-21T12:04:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:04:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[41097]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:04:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_count","val": "1"}]': finished 2024-09-21T12:04:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[41097]: osdmap e494: 8 total, 8 up, 8 in 2024-09-21T12:04:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:04:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:36 smithi193 ceph-mon[33333]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:04:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_count","val": "1"}]': finished 2024-09-21T12:04:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:36 smithi193 ceph-mon[33333]: osdmap e494: 8 total, 8 up, 8 in 2024-09-21T12:04:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[41097]: osdmap e495: 8 total, 8 up, 8 in 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[30293]: osdmap e495: 8 total, 8 up, 8 in 2024-09-21T12:04:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-21T12:04:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:37 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:04:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:37 smithi193 ceph-mon[33333]: osdmap e495: 8 total, 8 up, 8 in 2024-09-21T12:04:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "target_max_objects","val": "250"} : dispatch 2024-09-21T12:04:39.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[30293]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T12:04:39.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "target_max_objects","val": "250"}]': finished 2024-09-21T12:04:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[30293]: osdmap e496: 8 total, 8 up, 8 in 2024-09-21T12:04:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[41097]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T12:04:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "target_max_objects","val": "250"}]': finished 2024-09-21T12:04:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:38 smithi107 ceph-mon[41097]: osdmap e496: 8 total, 8 up, 8 in 2024-09-21T12:04:39.038 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:38 smithi193 ceph-mon[33333]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-21T12:04:39.058 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:39.059 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-98","var": "target_max_objects","val": "250"}]': finished 2024-09-21T12:04:39.059 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:38 smithi193 ceph-mon[33333]: osdmap e496: 8 total, 8 up, 8 in 2024-09-21T12:04:39.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:04:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:04:40.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:40.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:40.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:40.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:40.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:40.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:41.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[41097]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[41097]: osdmap e497: 8 total, 8 up, 8 in 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98"} : dispatch 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[30293]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[30293]: osdmap e497: 8 total, 8 up, 8 in 2024-09-21T12:04:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98"} : dispatch 2024-09-21T12:04:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:40 smithi193 ceph-mon[33333]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:04:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:40 smithi193 ceph-mon[33333]: osdmap e497: 8 total, 8 up, 8 in 2024-09-21T12:04:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98"} : dispatch 2024-09-21T12:04:42.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98"}]': finished 2024-09-21T12:04:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:41 smithi107 ceph-mon[41097]: osdmap e498: 8 total, 8 up, 8 in 2024-09-21T12:04:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98"}]': finished 2024-09-21T12:04:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:41 smithi107 ceph-mon[30293]: osdmap e498: 8 total, 8 up, 8 in 2024-09-21T12:04:42.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-98"}]': finished 2024-09-21T12:04:42.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:41 smithi193 ceph-mon[33333]: osdmap e498: 8 total, 8 up, 8 in 2024-09-21T12:04:42.711 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:04:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:04:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:04:43.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:42 smithi107 ceph-mon[30293]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:42 smithi107 ceph-mon[30293]: osdmap e499: 8 total, 8 up, 8 in 2024-09-21T12:04:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:42 smithi107 ceph-mon[41097]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:42 smithi107 ceph-mon[41097]: osdmap e499: 8 total, 8 up, 8 in 2024-09-21T12:04:43.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:42 smithi193 ceph-mon[33333]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:43.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:42 smithi193 ceph-mon[33333]: osdmap e499: 8 total, 8 up, 8 in 2024-09-21T12:04:44.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:43 smithi107 ceph-mon[30293]: osdmap e500: 8 total, 8 up, 8 in 2024-09-21T12:04:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:43 smithi107 ceph-mon[41097]: osdmap e500: 8 total, 8 up, 8 in 2024-09-21T12:04:44.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:44.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:43 smithi193 ceph-mon[33333]: osdmap e500: 8 total, 8 up, 8 in 2024-09-21T12:04:44.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-100","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:45.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[41097]: pgmap v757: 228 pgs: 20 creating+peering, 12 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:45.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[41097]: osdmap e501: 8 total, 8 up, 8 in 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[30293]: pgmap v757: 228 pgs: 20 creating+peering, 12 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[30293]: osdmap e501: 8 total, 8 up, 8 in 2024-09-21T12:04:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:04:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-mon[33333]: pgmap v757: 228 pgs: 20 creating+peering, 12 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-mon[33333]: osdmap e501: 8 total, 8 up, 8 in 2024-09-21T12:04:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:04:45.051 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore.transactions t=2024-09-21T12:04:44.629441024Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-21T12:04:45.051 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=cleanup t=2024-09-21T12:04:44.631493224Z level=info msg="Completed cleanup jobs" duration=12.811877ms 2024-09-21T12:04:45.051 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:04:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=plugins.update.checker t=2024-09-21T12:04:44.770323635Z level=info msg="Update check succeeded" duration=68.603684ms 2024-09-21T12:04:46.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:04:46.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[30293]: osdmap e502: 8 total, 8 up, 8 in 2024-09-21T12:04:46.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-100"} : dispatch 2024-09-21T12:04:46.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-100"}]': finished 2024-09-21T12:04:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:04:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[41097]: osdmap e502: 8 total, 8 up, 8 in 2024-09-21T12:04:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-100"} : dispatch 2024-09-21T12:04:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-100"}]': finished 2024-09-21T12:04:46.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:04:46.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:45 smithi193 ceph-mon[33333]: osdmap e502: 8 total, 8 up, 8 in 2024-09-21T12:04:46.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-100"} : dispatch 2024-09-21T12:04:46.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-6", "overlaypool": "test-rados-api-smithi107-77755-100"}]': finished 2024-09-21T12:04:47.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: pgmap v760: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: osdmap e503: 8 total, 8 up, 8 in 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-100", "mode": "writeback"} : dispatch 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-100", "mode": "writeback"}]': finished 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: osdmap e504: 8 total, 8 up, 8 in 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: pgmap v760: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: osdmap e503: 8 total, 8 up, 8 in 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-100", "mode": "writeback"} : dispatch 2024-09-21T12:04:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:04:47.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-100", "mode": "writeback"}]': finished 2024-09-21T12:04:47.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: osdmap e504: 8 total, 8 up, 8 in 2024-09-21T12:04:47.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:04:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: pgmap v760: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: osdmap e503: 8 total, 8 up, 8 in 2024-09-21T12:04:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-100", "mode": "writeback"} : dispatch 2024-09-21T12:04:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:04:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-100", "mode": "writeback"}]': finished 2024-09-21T12:04:47.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: osdmap e504: 8 total, 8 up, 8 in 2024-09-21T12:04:47.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:04:49.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[41097]: pgmap v763: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[41097]: osdmap e505: 8 total, 8 up, 8 in 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[30293]: pgmap v763: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[30293]: osdmap e505: 8 total, 8 up, 8 in 2024-09-21T12:04:49.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:04:49.041 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:48 smithi193 ceph-mon[33333]: pgmap v763: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-21T12:04:49.041 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:04:49.041 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:48 smithi193 ceph-mon[33333]: osdmap e505: 8 total, 8 up, 8 in 2024-09-21T12:04:49.041 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:04:49.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:04:50.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: osdmap e506: 8 total, 8 up, 8 in 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: osdmap e507: 8 total, 8 up, 8 in 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: osdmap e506: 8 total, 8 up, 8 in 2024-09-21T12:04:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:04:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:04:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:04:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: osdmap e507: 8 total, 8 up, 8 in 2024-09-21T12:04:50.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-21T12:04:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:04:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: osdmap e506: 8 total, 8 up, 8 in 2024-09-21T12:04:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:04:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:04:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:04:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:04:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: osdmap e507: 8 total, 8 up, 8 in 2024-09-21T12:04:50.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "min_read_recency_for_promote","val": "10000"} : dispatch 2024-09-21T12:04:51.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:50 smithi107 ceph-mon[30293]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-21T12:04:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:50 smithi107 ceph-mon[30293]: osdmap e508: 8 total, 8 up, 8 in 2024-09-21T12:04:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:50 smithi107 ceph-mon[41097]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-21T12:04:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:50 smithi107 ceph-mon[41097]: osdmap e508: 8 total, 8 up, 8 in 2024-09-21T12:04:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:50 smithi193 ceph-mon[33333]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-21T12:04:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:50 smithi193 ceph-mon[33333]: osdmap e508: 8 total, 8 up, 8 in 2024-09-21T12:04:52.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:04:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:04:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:04:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:52 smithi193 ceph-mon[33333]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:04:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:52 smithi193 ceph-mon[33333]: osdmap e509: 8 total, 8 up, 8 in 2024-09-21T12:04:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100"} : dispatch 2024-09-21T12:04:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:52 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:53.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[41097]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[41097]: osdmap e509: 8 total, 8 up, 8 in 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100"} : dispatch 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[30293]: pgmap v769: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"}]': finished 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[30293]: osdmap e509: 8 total, 8 up, 8 in 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100"} : dispatch 2024-09-21T12:04:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:52 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100"}]': finished 2024-09-21T12:04:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:53 smithi193 ceph-mon[33333]: osdmap e510: 8 total, 8 up, 8 in 2024-09-21T12:04:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:54.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100"}]': finished 2024-09-21T12:04:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:53 smithi107 ceph-mon[30293]: osdmap e510: 8 total, 8 up, 8 in 2024-09-21T12:04:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-100"}]': finished 2024-09-21T12:04:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:53 smithi107 ceph-mon[41097]: osdmap e510: 8 total, 8 up, 8 in 2024-09-21T12:04:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:04:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:54 smithi193 ceph-mon[33333]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:54 smithi193 ceph-mon[33333]: osdmap e511: 8 total, 8 up, 8 in 2024-09-21T12:04:55.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:54 smithi107 ceph-mon[41097]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:55.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:54 smithi107 ceph-mon[41097]: osdmap e511: 8 total, 8 up, 8 in 2024-09-21T12:04:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:54 smithi107 ceph-mon[30293]: pgmap v772: 228 pgs: 228 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:04:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:54 smithi107 ceph-mon[30293]: osdmap e511: 8 total, 8 up, 8 in 2024-09-21T12:04:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:55 smithi193 ceph-mon[33333]: osdmap e512: 8 total, 8 up, 8 in 2024-09-21T12:04:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:55 smithi193 ceph-mon[33333]: osdmap e513: 8 total, 8 up, 8 in 2024-09-21T12:04:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[41097]: osdmap e512: 8 total, 8 up, 8 in 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[41097]: osdmap e513: 8 total, 8 up, 8 in 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[30293]: osdmap e512: 8 total, 8 up, 8 in 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-102","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:04:56.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[30293]: osdmap e513: 8 total, 8 up, 8 in 2024-09-21T12:04:56.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd dump","format":"json"} : dispatch 2024-09-21T12:04:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:56 smithi193 ceph-mon[33333]: pgmap v775: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:04:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:04:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:56 smithi193 ceph-mon[33333]: osdmap e514: 8 total, 8 up, 8 in 2024-09-21T12:04:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:04:57.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[41097]: pgmap v775: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:57.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[41097]: osdmap e514: 8 total, 8 up, 8 in 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[30293]: pgmap v775: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "fingerprint_algorithm","val": "sha1"} : dispatch 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[30293]: osdmap e514: 8 total, 8 up, 8 in 2024-09-21T12:04:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_chunk_algorithm","val": "fastcdc"} : dispatch 2024-09-21T12:04:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:57 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:04:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:57 smithi193 ceph-mon[33333]: osdmap e515: 8 total, 8 up, 8 in 2024-09-21T12:04:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:04:58.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:58.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:04:58.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[41097]: osdmap e515: 8 total, 8 up, 8 in 2024-09-21T12:04:58.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:04:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:04:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-21T12:04:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[30293]: osdmap e515: 8 total, 8 up, 8 in 2024-09-21T12:04:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_cdc_chunk_size","val": "1024"} : dispatch 2024-09-21T12:04:59.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:58 smithi193 ceph-mon[33333]: pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:59.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:04:59.052 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:58 smithi193 ceph-mon[33333]: osdmap e516: 8 total, 8 up, 8 in 2024-09-21T12:04:59.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:58 smithi107 ceph-mon[41097]: pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:59.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:04:59.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:58 smithi107 ceph-mon[41097]: osdmap e516: 8 total, 8 up, 8 in 2024-09-21T12:04:59.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:58 smithi107 ceph-mon[30293]: pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:04:59.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-21T12:04:59.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:58 smithi107 ceph-mon[30293]: osdmap e516: 8 total, 8 up, 8 in 2024-09-21T12:04:59.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:04:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:00.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:05:00.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-102"} : dispatch 2024-09-21T12:05:00.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:04:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:05:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-102"} : dispatch 2024-09-21T12:05:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:04:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-6"} : dispatch 2024-09-21T12:05:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1969944610' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-6", "tierpool": "test-rados-api-smithi107-77755-102"} : dispatch 2024-09-21T12:05:00.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:04:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:01.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:00 smithi107 ceph-mon[41097]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:01.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:00 smithi107 ceph-mon[41097]: osdmap e517: 8 total, 8 up, 8 in 2024-09-21T12:05:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:00 smithi107 ceph-mon[30293]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:00 smithi107 ceph-mon[30293]: osdmap e517: 8 total, 8 up, 8 in 2024-09-21T12:05:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:00 smithi193 ceph-mon[33333]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:00 smithi193 ceph-mon[33333]: osdmap e517: 8 total, 8 up, 8 in 2024-09-21T12:05:02.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:01 smithi193 ceph-mon[33333]: osdmap e518: 8 total, 8 up, 8 in 2024-09-21T12:05:02.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:02.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:02.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:02.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[30293]: osdmap e518: 8 total, 8 up, 8 in 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[41097]: osdmap e518: 8 total, 8 up, 8 in 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:02.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:05:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:05:03.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[30293]: pgmap v784: 164 pgs: 164 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[30293]: osdmap e519: 8 total, 8 up, 8 in 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-77755-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[41097]: pgmap v784: 164 pgs: 164 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[41097]: osdmap e519: 8 total, 8 up, 8 in 2024-09-21T12:05:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-77755-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:03.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:02 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:02 smithi193 ceph-mon[33333]: pgmap v784: 164 pgs: 164 active+clean; 583 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:02 smithi193 ceph-mon[33333]: osdmap e519: 8 total, 8 up, 8 in 2024-09-21T12:05:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-77755-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:03.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:02 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:04.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:03 smithi107 ceph-mon[30293]: osdmap e520: 8 total, 8 up, 8 in 2024-09-21T12:05:04.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-77755-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:04.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:03 smithi107 ceph-mon[30293]: osdmap e521: 8 total, 8 up, 8 in 2024-09-21T12:05:04.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:03 smithi107 ceph-mon[41097]: osdmap e520: 8 total, 8 up, 8 in 2024-09-21T12:05:04.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-77755-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:04.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:03 smithi107 ceph-mon[41097]: osdmap e521: 8 total, 8 up, 8 in 2024-09-21T12:05:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:03 smithi193 ceph-mon[33333]: osdmap e520: 8 total, 8 up, 8 in 2024-09-21T12:05:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-77755-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:04.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:03 smithi193 ceph-mon[33333]: osdmap e521: 8 total, 8 up, 8 in 2024-09-21T12:05:05.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:04 smithi107 ceph-mon[41097]: pgmap v787: 164 pgs: 164 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:05.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:04 smithi107 ceph-mon[41097]: osdmap e522: 8 total, 8 up, 8 in 2024-09-21T12:05:05.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:04 smithi107 ceph-mon[30293]: pgmap v787: 164 pgs: 164 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:05.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:04 smithi107 ceph-mon[30293]: osdmap e522: 8 total, 8 up, 8 in 2024-09-21T12:05:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:04 smithi193 ceph-mon[33333]: pgmap v787: 164 pgs: 164 active+clean; 583 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:04 smithi193 ceph-mon[33333]: osdmap e522: 8 total, 8 up, 8 in 2024-09-21T12:05:07.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:06 smithi107 ceph-mon[30293]: pgmap v790: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:07.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:06 smithi107 ceph-mon[30293]: osdmap e523: 8 total, 8 up, 8 in 2024-09-21T12:05:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:06 smithi107 ceph-mon[41097]: pgmap v790: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:06 smithi107 ceph-mon[41097]: osdmap e523: 8 total, 8 up, 8 in 2024-09-21T12:05:07.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:07.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:06 smithi193 ceph-mon[33333]: pgmap v790: 172 pgs: 8 unknown, 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:07.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:06 smithi193 ceph-mon[33333]: osdmap e523: 8 total, 8 up, 8 in 2024-09-21T12:05:07.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:08.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[41097]: osdmap e524: 8 total, 8 up, 8 in 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[41097]: osdmap e525: 8 total, 8 up, 8 in 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[30293]: osdmap e524: 8 total, 8 up, 8 in 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[30293]: osdmap e525: 8 total, 8 up, 8 in 2024-09-21T12:05:08.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:08.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:08.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:07 smithi193 ceph-mon[33333]: osdmap e524: 8 total, 8 up, 8 in 2024-09-21T12:05:08.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:07 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:08.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:07 smithi193 ceph-mon[33333]: osdmap e525: 8 total, 8 up, 8 in 2024-09-21T12:05:08.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[41097]: pgmap v793: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[41097]: osdmap e526: 8 total, 8 up, 8 in 2024-09-21T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:09.155 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[30293]: pgmap v793: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:09.156 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:09.156 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:09.156 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[30293]: osdmap e526: 8 total, 8 up, 8 in 2024-09-21T12:05:09.156 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:08 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:09.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:05:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:08 smithi193 ceph-mon[33333]: pgmap v793: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:09.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:09.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:08 smithi193 ceph-mon[33333]: osdmap e526: 8 total, 8 up, 8 in 2024-09-21T12:05:09.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:08 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:10.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[30293]: osdmap e527: 8 total, 8 up, 8 in 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-107", "overlaypool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[41097]: osdmap e527: 8 total, 8 up, 8 in 2024-09-21T12:05:10.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-107", "overlaypool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:10.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:10.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:10.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:09 smithi193 ceph-mon[33333]: osdmap e527: 8 total, 8 up, 8 in 2024-09-21T12:05:10.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-107", "overlaypool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:11.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[30293]: pgmap v796: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:05:11.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-107", "overlaypool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[30293]: osdmap e528: 8 total, 8 up, 8 in 2024-09-21T12:05:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-107-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[41097]: pgmap v796: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:05:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-107", "overlaypool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[41097]: osdmap e528: 8 total, 8 up, 8 in 2024-09-21T12:05:11.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-107-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:10 smithi193 ceph-mon[33333]: pgmap v796: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:05:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-107", "overlaypool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:10 smithi193 ceph-mon[33333]: osdmap e528: 8 total, 8 up, 8 in 2024-09-21T12:05:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-107-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:12.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:12.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-107-cache", "mode": "writeback"}]': finished 2024-09-21T12:05:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[30293]: osdmap e529: 8 total, 8 up, 8 in 2024-09-21T12:05:12.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-107"} : dispatch 2024-09-21T12:05:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-107-cache", "mode": "writeback"}]': finished 2024-09-21T12:05:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[41097]: osdmap e529: 8 total, 8 up, 8 in 2024-09-21T12:05:12.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-107"} : dispatch 2024-09-21T12:05:12.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:11 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:12.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-107-cache", "mode": "writeback"}]': finished 2024-09-21T12:05:12.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:11 smithi193 ceph-mon[33333]: osdmap e529: 8 total, 8 up, 8 in 2024-09-21T12:05:12.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-107"} : dispatch 2024-09-21T12:05:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:05:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:05:13.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[30293]: pgmap v799: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:05:13.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-107"}]': finished 2024-09-21T12:05:13.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[30293]: osdmap e530: 8 total, 8 up, 8 in 2024-09-21T12:05:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[41097]: pgmap v799: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:05:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-107"}]': finished 2024-09-21T12:05:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[41097]: osdmap e530: 8 total, 8 up, 8 in 2024-09-21T12:05:13.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:12 smithi193 ceph-mon[33333]: pgmap v799: 236 pgs: 32 creating+peering, 204 active+clean; 583 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:05:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-107"}]': finished 2024-09-21T12:05:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:12 smithi193 ceph-mon[33333]: osdmap e530: 8 total, 8 up, 8 in 2024-09-21T12:05:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"} : dispatch 2024-09-21T12:05:14.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:13 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:14.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:13 smithi107 ceph-mon[30293]: osdmap e531: 8 total, 8 up, 8 in 2024-09-21T12:05:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:13 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:14.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:13 smithi107 ceph-mon[41097]: osdmap e531: 8 total, 8 up, 8 in 2024-09-21T12:05:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:13 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1691017865' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-107", "tierpool": "test-rados-api-smithi107-77755-107-cache"}]': finished 2024-09-21T12:05:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:13 smithi193 ceph-mon[33333]: osdmap e531: 8 total, 8 up, 8 in 2024-09-21T12:05:15.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:14 smithi107 ceph-mon[30293]: pgmap v802: 236 pgs: 6 creating+peering, 230 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:15.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:14 smithi107 ceph-mon[30293]: osdmap e532: 8 total, 8 up, 8 in 2024-09-21T12:05:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:14 smithi107 ceph-mon[41097]: pgmap v802: 236 pgs: 6 creating+peering, 230 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:14 smithi107 ceph-mon[41097]: osdmap e532: 8 total, 8 up, 8 in 2024-09-21T12:05:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:14 smithi193 ceph-mon[33333]: pgmap v802: 236 pgs: 6 creating+peering, 230 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:14 smithi193 ceph-mon[33333]: osdmap e532: 8 total, 8 up, 8 in 2024-09-21T12:05:17.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: pgmap v805: 204 pgs: 204 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: osdmap e533: 8 total, 8 up, 8 in 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: pgmap v805: 204 pgs: 204 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: osdmap e533: 8 total, 8 up, 8 in 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:17.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:16 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: pgmap v805: 204 pgs: 204 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: osdmap e533: 8 total, 8 up, 8 in 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:16 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:18.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[41097]: osdmap e534: 8 total, 8 up, 8 in 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[30293]: osdmap e534: 8 total, 8 up, 8 in 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:18.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:17 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:17 smithi193 ceph-mon[33333]: osdmap e534: 8 total, 8 up, 8 in 2024-09-21T12:05:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:17 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:19.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:18 smithi107 ceph-mon[41097]: pgmap v808: 172 pgs: 172 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:19.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:18 smithi107 ceph-mon[41097]: osdmap e535: 8 total, 8 up, 8 in 2024-09-21T12:05:19.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:18 smithi107 ceph-mon[30293]: pgmap v808: 172 pgs: 172 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:19.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:18 smithi107 ceph-mon[30293]: osdmap e535: 8 total, 8 up, 8 in 2024-09-21T12:05:19.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:05:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:18 smithi193 ceph-mon[33333]: pgmap v808: 172 pgs: 172 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:18 smithi193 ceph-mon[33333]: osdmap e535: 8 total, 8 up, 8 in 2024-09-21T12:05:20.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[30293]: osdmap e536: 8 total, 8 up, 8 in 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[30293]: osdmap e537: 8 total, 8 up, 8 in 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[41097]: osdmap e536: 8 total, 8 up, 8 in 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:19 smithi107 ceph-mon[41097]: osdmap e537: 8 total, 8 up, 8 in 2024-09-21T12:05:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:19 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:19 smithi193 ceph-mon[33333]: osdmap e536: 8 total, 8 up, 8 in 2024-09-21T12:05:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:19 smithi193 ceph-mon[33333]: osdmap e537: 8 total, 8 up, 8 in 2024-09-21T12:05:21.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: pgmap v811: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: osdmap e538: 8 total, 8 up, 8 in 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: pgmap v811: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:21.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:21.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:21.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: osdmap e538: 8 total, 8 up, 8 in 2024-09-21T12:05:21.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:21.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:20 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: pgmap v811: 180 pgs: 8 unknown, 172 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: osdmap e538: 8 total, 8 up, 8 in 2024-09-21T12:05:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:21.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:20 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:22.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:05:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:05:23.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[30293]: pgmap v814: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[30293]: osdmap e539: 8 total, 8 up, 8 in 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[41097]: pgmap v814: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[41097]: osdmap e539: 8 total, 8 up, 8 in 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:23.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:22 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:22 smithi193 ceph-mon[33333]: pgmap v814: 212 pgs: 40 unknown, 172 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:22 smithi193 ceph-mon[33333]: osdmap e539: 8 total, 8 up, 8 in 2024-09-21T12:05:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:22 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: osdmap e540: 8 total, 8 up, 8 in 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"}]': finished 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[30293]: osdmap e541: 8 total, 8 up, 8 in 2024-09-21T12:05:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: osdmap e540: 8 total, 8 up, 8 in 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"}]': finished 2024-09-21T12:05:24.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:23 smithi107 ceph-mon[41097]: osdmap e541: 8 total, 8 up, 8 in 2024-09-21T12:05:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-109", "overlaypool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: osdmap e540: 8 total, 8 up, 8 in 2024-09-21T12:05:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"} : dispatch 2024-09-21T12:05:24.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:24.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:24.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-109-cache", "mode": "writeback"}]': finished 2024-09-21T12:05:24.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:23 smithi193 ceph-mon[33333]: osdmap e541: 8 total, 8 up, 8 in 2024-09-21T12:05:25.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: pgmap v817: 212 pgs: 13 unknown, 199 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: pgmap v817: 212 pgs: 13 unknown, 199 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:05:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:05:25.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:05:25.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: pgmap v817: 212 pgs: 13 unknown, 199 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:05:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:05:26.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:05:26.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:05:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[41097]: osdmap e542: 8 total, 8 up, 8 in 2024-09-21T12:05:26.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:05:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:05:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:05:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[30293]: osdmap e542: 8 total, 8 up, 8 in 2024-09-21T12:05:26.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:25 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:05:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:05:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:05:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:25 smithi193 ceph-mon[33333]: osdmap e542: 8 total, 8 up, 8 in 2024-09-21T12:05:26.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:25 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:05:27.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[30293]: pgmap v820: 212 pgs: 212 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[30293]: osdmap e543: 8 total, 8 up, 8 in 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[41097]: pgmap v820: 212 pgs: 212 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[41097]: osdmap e543: 8 total, 8 up, 8 in 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:05:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:26 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:05:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:26 smithi193 ceph-mon[33333]: pgmap v820: 212 pgs: 212 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:05:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:26 smithi193 ceph-mon[33333]: osdmap e543: 8 total, 8 up, 8 in 2024-09-21T12:05:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:05:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:26 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:05:28.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:28.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[41097]: osdmap e544: 8 total, 8 up, 8 in 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[30293]: osdmap e544: 8 total, 8 up, 8 in 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-21T12:05:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:27 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-21T12:05:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:27 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:05:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:27 smithi193 ceph-mon[33333]: osdmap e544: 8 total, 8 up, 8 in 2024-09-21T12:05:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-21T12:05:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:27 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"} : dispatch 2024-09-21T12:05:29.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[30293]: pgmap v823: 212 pgs: 212 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[30293]: osdmap e545: 8 total, 8 up, 8 in 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[41097]: pgmap v823: 212 pgs: 212 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[41097]: osdmap e545: 8 total, 8 up, 8 in 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:28 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:29.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:28 smithi193 ceph-mon[33333]: pgmap v823: 212 pgs: 212 active+clean; 583 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:28 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-21T12:05:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:28 smithi193 ceph-mon[33333]: osdmap e545: 8 total, 8 up, 8 in 2024-09-21T12:05:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:28 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:30.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:30.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: osdmap e546: 8 total, 8 up, 8 in 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[41097]: osdmap e547: 8 total, 8 up, 8 in 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: osdmap e546: 8 total, 8 up, 8 in 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:30.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:29 smithi107 ceph-mon[30293]: osdmap e547: 8 total, 8 up, 8 in 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: osdmap e546: 8 total, 8 up, 8 in 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"} : dispatch 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-109", "tierpool": "test-rados-api-smithi107-77755-109-cache"}]': finished 2024-09-21T12:05:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:29 smithi193 ceph-mon[33333]: osdmap e547: 8 total, 8 up, 8 in 2024-09-21T12:05:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:30 smithi107 ceph-mon[30293]: pgmap v826: 212 pgs: 212 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:05:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:30 smithi107 ceph-mon[41097]: pgmap v826: 212 pgs: 212 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:05:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:30 smithi193 ceph-mon[33333]: pgmap v826: 212 pgs: 212 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:05:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:31 smithi107 ceph-mon[30293]: osdmap e548: 8 total, 8 up, 8 in 2024-09-21T12:05:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:31 smithi107 ceph-mon[30293]: osdmap e549: 8 total, 8 up, 8 in 2024-09-21T12:05:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:31 smithi107 ceph-mon[41097]: osdmap e548: 8 total, 8 up, 8 in 2024-09-21T12:05:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:31 smithi107 ceph-mon[41097]: osdmap e549: 8 total, 8 up, 8 in 2024-09-21T12:05:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:31 smithi193 ceph-mon[33333]: osdmap e548: 8 total, 8 up, 8 in 2024-09-21T12:05:32.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:31 smithi193 ceph-mon[33333]: osdmap e549: 8 total, 8 up, 8 in 2024-09-21T12:05:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:05:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:05:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: pgmap v829: 180 pgs: 180 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: osdmap e550: 8 total, 8 up, 8 in 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: pgmap v829: 180 pgs: 180 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: osdmap e550: 8 total, 8 up, 8 in 2024-09-21T12:05:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:32 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: pgmap v829: 180 pgs: 180 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: osdmap e550: 8 total, 8 up, 8 in 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2896410328' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:32 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"} : dispatch 2024-09-21T12:05:35.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[41097]: pgmap v832: 172 pgs: 172 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[41097]: osdmap e551: 8 total, 8 up, 8 in 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[30293]: pgmap v832: 172 pgs: 172 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[30293]: osdmap e551: 8 total, 8 up, 8 in 2024-09-21T12:05:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:34 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:34 smithi193 ceph-mon[33333]: pgmap v832: 172 pgs: 172 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T12:05:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-109"}]': finished 2024-09-21T12:05:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:34 smithi193 ceph-mon[33333]: osdmap e551: 8 total, 8 up, 8 in 2024-09-21T12:05:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:34 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:36.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[30293]: osdmap e552: 8 total, 8 up, 8 in 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[30293]: osdmap e553: 8 total, 8 up, 8 in 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[41097]: osdmap e552: 8 total, 8 up, 8 in 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[41097]: osdmap e553: 8 total, 8 up, 8 in 2024-09-21T12:05:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:35 smithi193 ceph-mon[33333]: osdmap e552: 8 total, 8 up, 8 in 2024-09-21T12:05:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:35 smithi193 ceph-mon[33333]: osdmap e553: 8 total, 8 up, 8 in 2024-09-21T12:05:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"} : dispatch 2024-09-21T12:05:37.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:36 smithi107 ceph-mon[41097]: pgmap v835: 164 pgs: 164 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T12:05:37.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:36 smithi107 ceph-mon[41097]: osdmap e554: 8 total, 8 up, 8 in 2024-09-21T12:05:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:36 smithi107 ceph-mon[30293]: pgmap v835: 164 pgs: 164 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T12:05:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:36 smithi107 ceph-mon[30293]: osdmap e554: 8 total, 8 up, 8 in 2024-09-21T12:05:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:36 smithi193 ceph-mon[33333]: pgmap v835: 164 pgs: 164 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-21T12:05:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3239385075' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-77755-104"}]': finished 2024-09-21T12:05:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:36 smithi193 ceph-mon[33333]: osdmap e554: 8 total, 8 up, 8 in 2024-09-21T12:05:38.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:38.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:38.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:38.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:38.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]} : dispatch 2024-09-21T12:05:39.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[30293]: pgmap v838: 164 pgs: 164 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[30293]: osdmap e555: 8 total, 8 up, 8 in 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[41097]: pgmap v838: 164 pgs: 164 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[41097]: osdmap e555: 8 total, 8 up, 8 in 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:39.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:05:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:38 smithi193 ceph-mon[33333]: pgmap v838: 164 pgs: 164 active+clean; 583 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-77755-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-21T12:05:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:38 smithi193 ceph-mon[33333]: osdmap e555: 8 total, 8 up, 8 in 2024-09-21T12:05:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:40.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[30293]: osdmap e556: 8 total, 8 up, 8 in 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[30293]: osdmap e557: 8 total, 8 up, 8 in 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[41097]: osdmap e556: 8 total, 8 up, 8 in 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:40.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:39 smithi107 ceph-mon[41097]: osdmap e557: 8 total, 8 up, 8 in 2024-09-21T12:05:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:39 smithi193 ceph-mon[33333]: osdmap e556: 8 total, 8 up, 8 in 2024-09-21T12:05:40.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-77755-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:40.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:39 smithi193 ceph-mon[33333]: osdmap e557: 8 total, 8 up, 8 in 2024-09-21T12:05:41.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:40 smithi107 ceph-mon[30293]: pgmap v841: 164 pgs: 164 active+clean; 583 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:40 smithi107 ceph-mon[30293]: osdmap e558: 8 total, 8 up, 8 in 2024-09-21T12:05:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:40 smithi107 ceph-mon[41097]: pgmap v841: 164 pgs: 164 active+clean; 583 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:40 smithi107 ceph-mon[41097]: osdmap e558: 8 total, 8 up, 8 in 2024-09-21T12:05:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:40 smithi193 ceph-mon[33333]: pgmap v841: 164 pgs: 164 active+clean; 583 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:40 smithi193 ceph-mon[33333]: osdmap e558: 8 total, 8 up, 8 in 2024-09-21T12:05:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-112","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:05:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:05:43.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[30293]: pgmap v844: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[30293]: osdmap e559: 8 total, 8 up, 8 in 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[41097]: pgmap v844: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[41097]: osdmap e559: 8 total, 8 up, 8 in 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:42 smithi193 ceph-mon[33333]: pgmap v844: 204 pgs: 40 unknown, 164 active+clean; 583 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:05:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:42 smithi193 ceph-mon[33333]: osdmap e559: 8 total, 8 up, 8 in 2024-09-21T12:05:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:42 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:44.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[30293]: osdmap e560: 8 total, 8 up, 8 in 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-112"} : dispatch 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-112"}]': finished 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[41097]: osdmap e560: 8 total, 8 up, 8 in 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-112"} : dispatch 2024-09-21T12:05:44.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-112"}]': finished 2024-09-21T12:05:44.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:05:44.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:43 smithi193 ceph-mon[33333]: osdmap e560: 8 total, 8 up, 8 in 2024-09-21T12:05:44.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-112"} : dispatch 2024-09-21T12:05:44.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-112"}]': finished 2024-09-21T12:05:45.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:44 smithi107 ceph-mon[41097]: pgmap v847: 204 pgs: 19 unknown, 185 active+clean; 583 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:44 smithi107 ceph-mon[41097]: osdmap e561: 8 total, 8 up, 8 in 2024-09-21T12:05:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:44 smithi107 ceph-mon[30293]: pgmap v847: 204 pgs: 19 unknown, 185 active+clean; 583 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:44 smithi107 ceph-mon[30293]: osdmap e561: 8 total, 8 up, 8 in 2024-09-21T12:05:45.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:44 smithi193 ceph-mon[33333]: pgmap v847: 204 pgs: 19 unknown, 185 active+clean; 583 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:44 smithi193 ceph-mon[33333]: osdmap e561: 8 total, 8 up, 8 in 2024-09-21T12:05:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:46.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:46.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:46 smithi193 ceph-mon[33333]: osdmap e562: 8 total, 8 up, 8 in 2024-09-21T12:05:46.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112"} : dispatch 2024-09-21T12:05:46.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112"}]': finished 2024-09-21T12:05:46.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:46 smithi193 ceph-mon[33333]: osdmap e563: 8 total, 8 up, 8 in 2024-09-21T12:05:46.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[30293]: osdmap e562: 8 total, 8 up, 8 in 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112"} : dispatch 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112"}]': finished 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[30293]: osdmap e563: 8 total, 8 up, 8 in 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[41097]: osdmap e562: 8 total, 8 up, 8 in 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112"} : dispatch 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-112"}]': finished 2024-09-21T12:05:46.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:46 smithi107 ceph-mon[41097]: osdmap e563: 8 total, 8 up, 8 in 2024-09-21T12:05:47.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:47 smithi193 ceph-mon[33333]: pgmap v850: 204 pgs: 204 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:05:47.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:47 smithi107 ceph-mon[30293]: pgmap v850: 204 pgs: 204 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:05:47.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:47 smithi107 ceph-mon[41097]: pgmap v850: 204 pgs: 204 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-21T12:05:48.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:48 smithi193 ceph-mon[33333]: osdmap e564: 8 total, 8 up, 8 in 2024-09-21T12:05:48.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:48 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:48.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:48 smithi193 ceph-mon[33333]: osdmap e565: 8 total, 8 up, 8 in 2024-09-21T12:05:48.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:48.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[41097]: osdmap e564: 8 total, 8 up, 8 in 2024-09-21T12:05:48.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:48.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[41097]: osdmap e565: 8 total, 8 up, 8 in 2024-09-21T12:05:48.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:48.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[30293]: osdmap e564: 8 total, 8 up, 8 in 2024-09-21T12:05:48.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:05:48.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[30293]: osdmap e565: 8 total, 8 up, 8 in 2024-09-21T12:05:48.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-114","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:49.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:05:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:49 smithi193 ceph-mon[33333]: pgmap v853: 172 pgs: 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:49 smithi193 ceph-mon[33333]: osdmap e566: 8 total, 8 up, 8 in 2024-09-21T12:05:49.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:49 smithi107 ceph-mon[41097]: pgmap v853: 172 pgs: 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:49.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:49.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:49 smithi107 ceph-mon[41097]: osdmap e566: 8 total, 8 up, 8 in 2024-09-21T12:05:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:49 smithi107 ceph-mon[30293]: pgmap v853: 172 pgs: 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:49.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:49 smithi107 ceph-mon[30293]: osdmap e566: 8 total, 8 up, 8 in 2024-09-21T12:05:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:50 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:05:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:50 smithi193 ceph-mon[33333]: osdmap e567: 8 total, 8 up, 8 in 2024-09-21T12:05:50.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-114"} : dispatch 2024-09-21T12:05:50.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:50.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:50.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[41097]: osdmap e567: 8 total, 8 up, 8 in 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-114"} : dispatch 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[30293]: osdmap e567: 8 total, 8 up, 8 in 2024-09-21T12:05:50.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-114"} : dispatch 2024-09-21T12:05:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:51 smithi193 ceph-mon[33333]: pgmap v856: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:05:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-114"}]': finished 2024-09-21T12:05:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:51 smithi193 ceph-mon[33333]: osdmap e568: 8 total, 8 up, 8 in 2024-09-21T12:05:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-114", "mode": "writeback"} : dispatch 2024-09-21T12:05:51.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[30293]: pgmap v856: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:05:51.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-114"}]': finished 2024-09-21T12:05:51.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[30293]: osdmap e568: 8 total, 8 up, 8 in 2024-09-21T12:05:51.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-114", "mode": "writeback"} : dispatch 2024-09-21T12:05:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[41097]: pgmap v856: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:05:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-114"}]': finished 2024-09-21T12:05:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[41097]: osdmap e568: 8 total, 8 up, 8 in 2024-09-21T12:05:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-114", "mode": "writeback"} : dispatch 2024-09-21T12:05:52.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:05:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:05:52.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:52.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-114", "mode": "writeback"}]': finished 2024-09-21T12:05:52.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-mon[41097]: osdmap e569: 8 total, 8 up, 8 in 2024-09-21T12:05:52.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:52.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-114", "mode": "writeback"}]': finished 2024-09-21T12:05:52.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:52 smithi107 ceph-mon[30293]: osdmap e569: 8 total, 8 up, 8 in 2024-09-21T12:05:52.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:52 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:05:52.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-114", "mode": "writeback"}]': finished 2024-09-21T12:05:52.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:52 smithi193 ceph-mon[33333]: osdmap e569: 8 total, 8 up, 8 in 2024-09-21T12:05:53.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:53 smithi107 ceph-mon[30293]: pgmap v859: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:05:53.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:53.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:53 smithi107 ceph-mon[41097]: pgmap v859: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:05:53.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:53 smithi193 ceph-mon[33333]: pgmap v859: 204 pgs: 32 creating+peering, 172 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T12:05:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[30293]: osdmap e570: 8 total, 8 up, 8 in 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114"} : dispatch 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[41097]: osdmap e570: 8 total, 8 up, 8 in 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114"} : dispatch 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:05:54.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:54 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:05:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:54 smithi193 ceph-mon[33333]: osdmap e570: 8 total, 8 up, 8 in 2024-09-21T12:05:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114"} : dispatch 2024-09-21T12:05:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:54 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:05:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:54 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:05:55.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[30293]: pgmap v862: 204 pgs: 16 creating+peering, 188 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:05:55.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114"}]': finished 2024-09-21T12:05:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[30293]: osdmap e571: 8 total, 8 up, 8 in 2024-09-21T12:05:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[41097]: pgmap v862: 204 pgs: 16 creating+peering, 188 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:05:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114"}]': finished 2024-09-21T12:05:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:55 smithi107 ceph-mon[41097]: osdmap e571: 8 total, 8 up, 8 in 2024-09-21T12:05:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:55 smithi193 ceph-mon[33333]: pgmap v862: 204 pgs: 16 creating+peering, 188 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T12:05:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:55 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:05:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-114"}]': finished 2024-09-21T12:05:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:55 smithi193 ceph-mon[33333]: osdmap e571: 8 total, 8 up, 8 in 2024-09-21T12:05:56.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:56 smithi107 ceph-mon[30293]: osdmap e572: 8 total, 8 up, 8 in 2024-09-21T12:05:56.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:56 smithi107 ceph-mon[41097]: osdmap e572: 8 total, 8 up, 8 in 2024-09-21T12:05:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:56 smithi193 ceph-mon[33333]: osdmap e572: 8 total, 8 up, 8 in 2024-09-21T12:05:57.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:57 smithi107 ceph-mon[30293]: pgmap v865: 172 pgs: 172 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:57 smithi107 ceph-mon[30293]: osdmap e573: 8 total, 8 up, 8 in 2024-09-21T12:05:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:57 smithi107 ceph-mon[41097]: pgmap v865: 172 pgs: 172 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:57 smithi107 ceph-mon[41097]: osdmap e573: 8 total, 8 up, 8 in 2024-09-21T12:05:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:57 smithi193 ceph-mon[33333]: pgmap v865: 172 pgs: 172 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:05:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:57 smithi193 ceph-mon[33333]: osdmap e573: 8 total, 8 up, 8 in 2024-09-21T12:05:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-116","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:05:58.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:58.506 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:58 smithi107 ceph-mon[41097]: osdmap e574: 8 total, 8 up, 8 in 2024-09-21T12:05:58.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:58.506 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:58 smithi107 ceph-mon[30293]: osdmap e574: 8 total, 8 up, 8 in 2024-09-21T12:05:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:05:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:58 smithi193 ceph-mon[33333]: osdmap e574: 8 total, 8 up, 8 in 2024-09-21T12:05:59.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:59 smithi107 ceph-mon[30293]: pgmap v868: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:59 smithi107 ceph-mon[30293]: osdmap e575: 8 total, 8 up, 8 in 2024-09-21T12:05:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:05:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:59.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:59 smithi107 ceph-mon[41097]: pgmap v868: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:59.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:59 smithi107 ceph-mon[41097]: osdmap e575: 8 total, 8 up, 8 in 2024-09-21T12:05:59.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:05:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:05:59.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:05:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:05:59.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:59 smithi193 ceph-mon[33333]: pgmap v868: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:05:59.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:59 smithi193 ceph-mon[33333]: osdmap e575: 8 total, 8 up, 8 in 2024-09-21T12:05:59.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:05:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:00.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:00.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: osdmap e576: 8 total, 8 up, 8 in 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-116"} : dispatch 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-116"}]': finished 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: osdmap e577: 8 total, 8 up, 8 in 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-116", "mode": "writeback"} : dispatch 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: osdmap e576: 8 total, 8 up, 8 in 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-116"} : dispatch 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-116"}]': finished 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: osdmap e577: 8 total, 8 up, 8 in 2024-09-21T12:06:00.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-116", "mode": "writeback"} : dispatch 2024-09-21T12:06:00.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:00.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:00.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: osdmap e576: 8 total, 8 up, 8 in 2024-09-21T12:06:00.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-116"} : dispatch 2024-09-21T12:06:00.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-116"}]': finished 2024-09-21T12:06:00.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: osdmap e577: 8 total, 8 up, 8 in 2024-09-21T12:06:00.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-116", "mode": "writeback"} : dispatch 2024-09-21T12:06:01.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[30293]: pgmap v871: 204 pgs: 204 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-116", "mode": "writeback"}]': finished 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[30293]: osdmap e578: 8 total, 8 up, 8 in 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[41097]: pgmap v871: 204 pgs: 204 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-116", "mode": "writeback"}]': finished 2024-09-21T12:06:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:01 smithi107 ceph-mon[41097]: osdmap e578: 8 total, 8 up, 8 in 2024-09-21T12:06:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:01 smithi193 ceph-mon[33333]: pgmap v871: 204 pgs: 204 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:01 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-116", "mode": "writeback"}]': finished 2024-09-21T12:06:01.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:01 smithi193 ceph-mon[33333]: osdmap e578: 8 total, 8 up, 8 in 2024-09-21T12:06:02.451 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.4"} : dispatch 2024-09-21T12:06:02.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:02 smithi193 ceph-mon[33333]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.4"}]: dispatch 2024-09-21T12:06:02.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:02 smithi193 ceph-mon[33333]: 325.4 scrub starts 2024-09-21T12:06:02.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:02 smithi193 ceph-mon[33333]: 325.4 scrub ok 2024-09-21T12:06:02.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:06:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.4"} : dispatch 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[30293]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.4"}]: dispatch 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[30293]: 325.4 scrub starts 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[30293]: 325.4 scrub ok 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "pg scrub", "pgid": "325.4"} : dispatch 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[41097]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.4"}]: dispatch 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[41097]: 325.4 scrub starts 2024-09-21T12:06:02.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:02 smithi107 ceph-mon[41097]: 325.4 scrub ok 2024-09-21T12:06:03.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:03 smithi107 ceph-mon[30293]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:03.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:03 smithi107 ceph-mon[41097]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:03.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:03 smithi193 ceph-mon[33333]: pgmap v874: 204 pgs: 204 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:05.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:05 smithi107 ceph-mon[41097]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T12:06:05.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:05 smithi107 ceph-mon[30293]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T12:06:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:05 smithi193 ceph-mon[33333]: pgmap v875: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-21T12:06:07.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:07 smithi107 ceph-mon[41097]: pgmap v876: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:06:07.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:07 smithi107 ceph-mon[30293]: pgmap v876: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:06:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:07 smithi193 ceph-mon[33333]: pgmap v876: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:06:09.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:09 smithi107 ceph-mon[30293]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-09-21T12:06:09.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:09 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:09 smithi107 ceph-mon[41097]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-09-21T12:06:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:09 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:09.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:06:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:06:09.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:09 smithi193 ceph-mon[33333]: pgmap v877: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-09-21T12:06:09.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:09 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:10.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:10.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:10.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:11.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:11 smithi107 ceph-mon[30293]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:06:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:11 smithi107 ceph-mon[41097]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:06:11.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:11 smithi193 ceph-mon[33333]: pgmap v878: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:06:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:06:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:06:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:06:13.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:13 smithi107 ceph-mon[30293]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:13.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:13 smithi107 ceph-mon[41097]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:13.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:13 smithi193 ceph-mon[33333]: pgmap v879: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:15.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:15 smithi107 ceph-mon[41097]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:06:15.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:15 smithi107 ceph-mon[30293]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:06:15.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:15 smithi193 ceph-mon[33333]: pgmap v880: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-21T12:06:17.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:17 smithi107 ceph-mon[30293]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:06:17.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:17 smithi107 ceph-mon[30293]: osdmap e579: 8 total, 8 up, 8 in 2024-09-21T12:06:17.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:17.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:17 smithi107 ceph-mon[41097]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:06:17.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:17 smithi107 ceph-mon[41097]: osdmap e579: 8 total, 8 up, 8 in 2024-09-21T12:06:17.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:17 smithi193 ceph-mon[33333]: pgmap v881: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:06:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:17 smithi193 ceph-mon[33333]: osdmap e579: 8 total, 8 up, 8 in 2024-09-21T12:06:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:18.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:18.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[30293]: osdmap e580: 8 total, 8 up, 8 in 2024-09-21T12:06:18.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116"} : dispatch 2024-09-21T12:06:18.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:18.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116"}]': finished 2024-09-21T12:06:18.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:18.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[41097]: osdmap e580: 8 total, 8 up, 8 in 2024-09-21T12:06:18.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116"} : dispatch 2024-09-21T12:06:18.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:18.505 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116"}]': finished 2024-09-21T12:06:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:18 smithi193 ceph-mon[33333]: osdmap e580: 8 total, 8 up, 8 in 2024-09-21T12:06:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116"} : dispatch 2024-09-21T12:06:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:18 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:18.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-116"}]': finished 2024-09-21T12:06:19.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:19 smithi107 ceph-mon[30293]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:06:19.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:19 smithi107 ceph-mon[30293]: osdmap e581: 8 total, 8 up, 8 in 2024-09-21T12:06:19.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:19.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:19 smithi107 ceph-mon[41097]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:06:19.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:19 smithi107 ceph-mon[41097]: osdmap e581: 8 total, 8 up, 8 in 2024-09-21T12:06:19.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:19.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:06:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:06:19.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:19 smithi193 ceph-mon[33333]: pgmap v884: 204 pgs: 204 active+clean; 583 KiB data, 588 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:06:19.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:19 smithi193 ceph-mon[33333]: osdmap e581: 8 total, 8 up, 8 in 2024-09-21T12:06:19.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:20.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[30293]: osdmap e582: 8 total, 8 up, 8 in 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[30293]: pgmap v887: 172 pgs: 172 active+clean; 583 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[30293]: osdmap e583: 8 total, 8 up, 8 in 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[41097]: osdmap e582: 8 total, 8 up, 8 in 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[41097]: pgmap v887: 172 pgs: 172 active+clean; 583 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[41097]: osdmap e583: 8 total, 8 up, 8 in 2024-09-21T12:06:20.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:20 smithi193 ceph-mon[33333]: osdmap e582: 8 total, 8 up, 8 in 2024-09-21T12:06:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:20 smithi193 ceph-mon[33333]: pgmap v887: 172 pgs: 172 active+clean; 583 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-21T12:06:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:20 smithi193 ceph-mon[33333]: osdmap e583: 8 total, 8 up, 8 in 2024-09-21T12:06:20.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-118","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:22.503 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:06:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:06:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-mon[30293]: osdmap e584: 8 total, 8 up, 8 in 2024-09-21T12:06:22.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-mon[30293]: pgmap v890: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-21T12:06:22.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:22.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-mon[41097]: osdmap e584: 8 total, 8 up, 8 in 2024-09-21T12:06:22.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:22 smithi107 ceph-mon[41097]: pgmap v890: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-21T12:06:22.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:22.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:22 smithi193 ceph-mon[33333]: osdmap e584: 8 total, 8 up, 8 in 2024-09-21T12:06:22.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:22 smithi193 ceph-mon[33333]: pgmap v890: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 606 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-21T12:06:23.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[30293]: osdmap e585: 8 total, 8 up, 8 in 2024-09-21T12:06:23.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[30293]: osdmap e586: 8 total, 8 up, 8 in 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-118"} : dispatch 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[41097]: osdmap e585: 8 total, 8 up, 8 in 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[41097]: osdmap e586: 8 total, 8 up, 8 in 2024-09-21T12:06:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-118"} : dispatch 2024-09-21T12:06:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:23 smithi193 ceph-mon[33333]: osdmap e585: 8 total, 8 up, 8 in 2024-09-21T12:06:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:23 smithi193 ceph-mon[33333]: osdmap e586: 8 total, 8 up, 8 in 2024-09-21T12:06:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-118"} : dispatch 2024-09-21T12:06:24.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-118"}]': finished 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[41097]: osdmap e587: 8 total, 8 up, 8 in 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-118", "mode": "writeback"} : dispatch 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[41097]: pgmap v894: 204 pgs: 9 unknown, 195 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 616 B/s wr, 2 op/s 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-118"}]': finished 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[30293]: osdmap e587: 8 total, 8 up, 8 in 2024-09-21T12:06:24.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-118", "mode": "writeback"} : dispatch 2024-09-21T12:06:24.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:24 smithi107 ceph-mon[30293]: pgmap v894: 204 pgs: 9 unknown, 195 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 616 B/s wr, 2 op/s 2024-09-21T12:06:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:24 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-118"}]': finished 2024-09-21T12:06:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:24 smithi193 ceph-mon[33333]: osdmap e587: 8 total, 8 up, 8 in 2024-09-21T12:06:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-118", "mode": "writeback"} : dispatch 2024-09-21T12:06:24.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:24 smithi193 ceph-mon[33333]: pgmap v894: 204 pgs: 9 unknown, 195 active+clean; 583 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 616 B/s wr, 2 op/s 2024-09-21T12:06:25.706 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:25 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:25.706 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:25 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-118", "mode": "writeback"}]': finished 2024-09-21T12:06:25.706 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:25 smithi193 ceph-mon[33333]: osdmap e588: 8 total, 8 up, 8 in 2024-09-21T12:06:25.706 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:25 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:06:25.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-118", "mode": "writeback"}]': finished 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[30293]: osdmap e588: 8 total, 8 up, 8 in 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-118", "mode": "writeback"}]': finished 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[41097]: osdmap e588: 8 total, 8 up, 8 in 2024-09-21T12:06:25.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:25 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:06:26.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:26 smithi107 ceph-mon[30293]: osdmap e589: 8 total, 8 up, 8 in 2024-09-21T12:06:26.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:26.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:26 smithi107 ceph-mon[30293]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 617 B/s wr, 2 op/s 2024-09-21T12:06:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:26 smithi107 ceph-mon[41097]: osdmap e589: 8 total, 8 up, 8 in 2024-09-21T12:06:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:26.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:26 smithi107 ceph-mon[41097]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 617 B/s wr, 2 op/s 2024-09-21T12:06:26.775 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:26 smithi193 ceph-mon[33333]: osdmap e589: 8 total, 8 up, 8 in 2024-09-21T12:06:26.775 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:26.775 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:26 smithi193 ceph-mon[33333]: pgmap v897: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 617 B/s wr, 2 op/s 2024-09-21T12:06:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:27 smithi107 ceph-mon[30293]: osdmap e590: 8 total, 8 up, 8 in 2024-09-21T12:06:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118"} : dispatch 2024-09-21T12:06:27.734 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:27.734 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:27 smithi107 ceph-mon[41097]: osdmap e590: 8 total, 8 up, 8 in 2024-09-21T12:06:27.734 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118"} : dispatch 2024-09-21T12:06:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:27 smithi193 ceph-mon[33333]: osdmap e590: 8 total, 8 up, 8 in 2024-09-21T12:06:27.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118"} : dispatch 2024-09-21T12:06:28.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118"}]': finished 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[30293]: osdmap e591: 8 total, 8 up, 8 in 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[30293]: pgmap v900: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118"}]': finished 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[41097]: osdmap e591: 8 total, 8 up, 8 in 2024-09-21T12:06:28.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[41097]: pgmap v900: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:28 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:28 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-118"}]': finished 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:28 smithi193 ceph-mon[33333]: osdmap e591: 8 total, 8 up, 8 in 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:28 smithi193 ceph-mon[33333]: pgmap v900: 204 pgs: 204 active+clean; 583 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:28 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:28.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:28 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:29.429 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:06:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:06:29.429 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:29 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:29.429 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:29 smithi193 ceph-mon[33333]: osdmap e592: 8 total, 8 up, 8 in 2024-09-21T12:06:29.429 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:29.429 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:29 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:29.429 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:29 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:29.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:29.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[41097]: osdmap e592: 8 total, 8 up, 8 in 2024-09-21T12:06:29.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:29.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:29.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[30293]: osdmap e592: 8 total, 8 up, 8 in 2024-09-21T12:06:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:29.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:29 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[30293]: osdmap e593: 8 total, 8 up, 8 in 2024-09-21T12:06:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:30.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[30293]: pgmap v903: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[41097]: osdmap e593: 8 total, 8 up, 8 in 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[41097]: pgmap v903: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:06:30.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:30 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:30 smithi193 ceph-mon[33333]: osdmap e593: 8 total, 8 up, 8 in 2024-09-21T12:06:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-120","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:30 smithi193 ceph-mon[33333]: pgmap v903: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:30 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:06:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:30 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:06:30.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:30 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:31 smithi107 ceph-mon[30293]: osdmap e594: 8 total, 8 up, 8 in 2024-09-21T12:06:31.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:31.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:31.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:31 smithi107 ceph-mon[41097]: osdmap e594: 8 total, 8 up, 8 in 2024-09-21T12:06:31.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:31.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:31 smithi193 ceph-mon[33333]: osdmap e594: 8 total, 8 up, 8 in 2024-09-21T12:06:31.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:32.716 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:32.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: osdmap e595: 8 total, 8 up, 8 in 2024-09-21T12:06:32.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-120"} : dispatch 2024-09-21T12:06:32.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: pgmap v906: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:32.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-120"}]': finished 2024-09-21T12:06:32.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: osdmap e596: 8 total, 8 up, 8 in 2024-09-21T12:06:32.717 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-120", "mode": "writeback"} : dispatch 2024-09-21T12:06:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:06:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: osdmap e595: 8 total, 8 up, 8 in 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-120"} : dispatch 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: pgmap v906: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-120"}]': finished 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: osdmap e596: 8 total, 8 up, 8 in 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-120", "mode": "writeback"} : dispatch 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: osdmap e595: 8 total, 8 up, 8 in 2024-09-21T12:06:32.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-120"} : dispatch 2024-09-21T12:06:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: pgmap v906: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-120"}]': finished 2024-09-21T12:06:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: osdmap e596: 8 total, 8 up, 8 in 2024-09-21T12:06:32.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-120", "mode": "writeback"} : dispatch 2024-09-21T12:06:33.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:33 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-120", "mode": "writeback"}]': finished 2024-09-21T12:06:33.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:33 smithi107 ceph-mon[30293]: osdmap e597: 8 total, 8 up, 8 in 2024-09-21T12:06:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:33 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-120", "mode": "writeback"}]': finished 2024-09-21T12:06:33.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:33 smithi107 ceph-mon[41097]: osdmap e597: 8 total, 8 up, 8 in 2024-09-21T12:06:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:33 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-120", "mode": "writeback"}]': finished 2024-09-21T12:06:33.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:33 smithi193 ceph-mon[33333]: osdmap e597: 8 total, 8 up, 8 in 2024-09-21T12:06:34.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:34.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:34 smithi107 ceph-mon[30293]: pgmap v909: 204 pgs: 11 unknown, 193 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:06:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:34.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:34 smithi107 ceph-mon[41097]: pgmap v909: 204 pgs: 11 unknown, 193 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:06:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:34 smithi193 ceph-mon[33333]: pgmap v909: 204 pgs: 11 unknown, 193 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:06:35.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:35.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:35 smithi107 ceph-mon[41097]: osdmap e598: 8 total, 8 up, 8 in 2024-09-21T12:06:35.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120"} : dispatch 2024-09-21T12:06:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:35 smithi107 ceph-mon[30293]: osdmap e598: 8 total, 8 up, 8 in 2024-09-21T12:06:35.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120"} : dispatch 2024-09-21T12:06:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:35 smithi193 ceph-mon[33333]: osdmap e598: 8 total, 8 up, 8 in 2024-09-21T12:06:35.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120"} : dispatch 2024-09-21T12:06:36.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:36.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120"}]': finished 2024-09-21T12:06:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[30293]: osdmap e599: 8 total, 8 up, 8 in 2024-09-21T12:06:36.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[30293]: pgmap v912: 204 pgs: 204 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120"}]': finished 2024-09-21T12:06:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[41097]: osdmap e599: 8 total, 8 up, 8 in 2024-09-21T12:06:36.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:36 smithi107 ceph-mon[41097]: pgmap v912: 204 pgs: 204 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:36 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-120"}]': finished 2024-09-21T12:06:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:36 smithi193 ceph-mon[33333]: osdmap e599: 8 total, 8 up, 8 in 2024-09-21T12:06:36.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:36 smithi193 ceph-mon[33333]: pgmap v912: 204 pgs: 204 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:37.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:37 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:37.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:37 smithi107 ceph-mon[30293]: osdmap e600: 8 total, 8 up, 8 in 2024-09-21T12:06:37.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:37 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:37.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:37 smithi107 ceph-mon[41097]: osdmap e600: 8 total, 8 up, 8 in 2024-09-21T12:06:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:37 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:37.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:37 smithi193 ceph-mon[33333]: osdmap e600: 8 total, 8 up, 8 in 2024-09-21T12:06:38.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[41097]: osdmap e601: 8 total, 8 up, 8 in 2024-09-21T12:06:38.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[41097]: pgmap v915: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:38.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[30293]: osdmap e601: 8 total, 8 up, 8 in 2024-09-21T12:06:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[30293]: pgmap v915: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:38.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:38 smithi193 ceph-mon[33333]: osdmap e601: 8 total, 8 up, 8 in 2024-09-21T12:06:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-122","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:38 smithi193 ceph-mon[33333]: pgmap v915: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:38.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:39.497 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:06:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:06:39.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[41097]: osdmap e602: 8 total, 8 up, 8 in 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[30293]: osdmap e602: 8 total, 8 up, 8 in 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:39.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:39 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:39 smithi193 ceph-mon[33333]: osdmap e602: 8 total, 8 up, 8 in 2024-09-21T12:06:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:39.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:39 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:40 smithi193 ceph-mon[33333]: pgmap v917: 204 pgs: 26 creating+peering, 178 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-09-21T12:06:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:40 smithi193 ceph-mon[33333]: osdmap e603: 8 total, 8 up, 8 in 2024-09-21T12:06:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-122"} : dispatch 2024-09-21T12:06:40.882 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[41097]: pgmap v917: 204 pgs: 26 creating+peering, 178 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[41097]: osdmap e603: 8 total, 8 up, 8 in 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-122"} : dispatch 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[30293]: pgmap v917: 204 pgs: 26 creating+peering, 178 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[30293]: osdmap e603: 8 total, 8 up, 8 in 2024-09-21T12:06:40.883 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-122"} : dispatch 2024-09-21T12:06:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-122"}]': finished 2024-09-21T12:06:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:41 smithi193 ceph-mon[33333]: osdmap e604: 8 total, 8 up, 8 in 2024-09-21T12:06:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-122", "mode": "writeback"} : dispatch 2024-09-21T12:06:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:41 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-122", "mode": "writeback"}]': finished 2024-09-21T12:06:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:41 smithi193 ceph-mon[33333]: osdmap e605: 8 total, 8 up, 8 in 2024-09-21T12:06:42.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-122"}]': finished 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[41097]: osdmap e604: 8 total, 8 up, 8 in 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-122", "mode": "writeback"} : dispatch 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-122", "mode": "writeback"}]': finished 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[41097]: osdmap e605: 8 total, 8 up, 8 in 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-122"}]': finished 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[30293]: osdmap e604: 8 total, 8 up, 8 in 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-122", "mode": "writeback"} : dispatch 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-122", "mode": "writeback"}]': finished 2024-09-21T12:06:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:41 smithi107 ceph-mon[30293]: osdmap e605: 8 total, 8 up, 8 in 2024-09-21T12:06:42.523 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:06:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:06:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:42 smithi193 ceph-mon[33333]: pgmap v920: 204 pgs: 26 creating+peering, 178 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-21T12:06:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:42 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:43.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-mon[41097]: pgmap v920: 204 pgs: 26 creating+peering, 178 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-21T12:06:43.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:43.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-mon[30293]: pgmap v920: 204 pgs: 26 creating+peering, 178 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-21T12:06:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:43.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:42 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:06:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:43 smithi193 ceph-mon[33333]: osdmap e606: 8 total, 8 up, 8 in 2024-09-21T12:06:43.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:43 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122"} : dispatch 2024-09-21T12:06:44.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:44.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:43 smithi107 ceph-mon[41097]: osdmap e606: 8 total, 8 up, 8 in 2024-09-21T12:06:44.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:43 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122"} : dispatch 2024-09-21T12:06:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:43 smithi107 ceph-mon[30293]: osdmap e606: 8 total, 8 up, 8 in 2024-09-21T12:06:44.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:43 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122"} : dispatch 2024-09-21T12:06:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:44 smithi193 ceph-mon[33333]: pgmap v923: 204 pgs: 204 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:44 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:44 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122"}]': finished 2024-09-21T12:06:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:44 smithi193 ceph-mon[33333]: osdmap e607: 8 total, 8 up, 8 in 2024-09-21T12:06:45.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[30293]: pgmap v923: 204 pgs: 204 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122"}]': finished 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[30293]: osdmap e607: 8 total, 8 up, 8 in 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[41097]: pgmap v923: 204 pgs: 204 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-122"}]': finished 2024-09-21T12:06:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:44 smithi107 ceph-mon[41097]: osdmap e607: 8 total, 8 up, 8 in 2024-09-21T12:06:46.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:45 smithi107 ceph-mon[30293]: osdmap e608: 8 total, 8 up, 8 in 2024-09-21T12:06:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:45 smithi107 ceph-mon[41097]: osdmap e608: 8 total, 8 up, 8 in 2024-09-21T12:06:46.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:45 smithi193 ceph-mon[33333]: osdmap e608: 8 total, 8 up, 8 in 2024-09-21T12:06:47.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:46 smithi107 ceph-mon[30293]: pgmap v926: 172 pgs: 172 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:46 smithi107 ceph-mon[30293]: osdmap e609: 8 total, 8 up, 8 in 2024-09-21T12:06:47.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:46 smithi107 ceph-mon[41097]: pgmap v926: 172 pgs: 172 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:46 smithi107 ceph-mon[41097]: osdmap e609: 8 total, 8 up, 8 in 2024-09-21T12:06:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:46 smithi193 ceph-mon[33333]: pgmap v926: 172 pgs: 172 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:06:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:46 smithi193 ceph-mon[33333]: osdmap e609: 8 total, 8 up, 8 in 2024-09-21T12:06:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-124","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:48.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:48.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:47 smithi107 ceph-mon[30293]: osdmap e610: 8 total, 8 up, 8 in 2024-09-21T12:06:48.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:48.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:47 smithi107 ceph-mon[41097]: osdmap e610: 8 total, 8 up, 8 in 2024-09-21T12:06:48.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:48.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:47 smithi193 ceph-mon[33333]: osdmap e610: 8 total, 8 up, 8 in 2024-09-21T12:06:49.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:48 smithi107 ceph-mon[30293]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:49.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:48 smithi107 ceph-mon[30293]: osdmap e611: 8 total, 8 up, 8 in 2024-09-21T12:06:49.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:48 smithi107 ceph-mon[41097]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:48 smithi107 ceph-mon[41097]: osdmap e611: 8 total, 8 up, 8 in 2024-09-21T12:06:49.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:48 smithi193 ceph-mon[33333]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:48 smithi193 ceph-mon[33333]: osdmap e611: 8 total, 8 up, 8 in 2024-09-21T12:06:49.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:49.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:06:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:06:50.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:50.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[41097]: osdmap e612: 8 total, 8 up, 8 in 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-124"} : dispatch 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-124"}]': finished 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[30293]: osdmap e612: 8 total, 8 up, 8 in 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-124"} : dispatch 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:50.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-124"}]': finished 2024-09-21T12:06:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:49 smithi193 ceph-mon[33333]: osdmap e612: 8 total, 8 up, 8 in 2024-09-21T12:06:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-124"} : dispatch 2024-09-21T12:06:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:49 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:06:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-124"}]': finished 2024-09-21T12:06:50.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[41097]: pgmap v932: 204 pgs: 204 active+clean; 583 KiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[41097]: osdmap e613: 8 total, 8 up, 8 in 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-124", "mode": "writeback"} : dispatch 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-124", "mode": "writeback"}]': finished 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[41097]: osdmap e614: 8 total, 8 up, 8 in 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[30293]: pgmap v932: 204 pgs: 204 active+clean; 583 KiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[30293]: osdmap e613: 8 total, 8 up, 8 in 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-124", "mode": "writeback"} : dispatch 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:50.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-124", "mode": "writeback"}]': finished 2024-09-21T12:06:50.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:50 smithi107 ceph-mon[30293]: osdmap e614: 8 total, 8 up, 8 in 2024-09-21T12:06:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:50 smithi193 ceph-mon[33333]: pgmap v932: 204 pgs: 204 active+clean; 583 KiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:50 smithi193 ceph-mon[33333]: osdmap e613: 8 total, 8 up, 8 in 2024-09-21T12:06:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-124", "mode": "writeback"} : dispatch 2024-09-21T12:06:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:50 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:06:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-124", "mode": "writeback"}]': finished 2024-09-21T12:06:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:50 smithi193 ceph-mon[33333]: osdmap e614: 8 total, 8 up, 8 in 2024-09-21T12:06:52.597 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:06:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:06:53.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-mon[41097]: pgmap v935: 204 pgs: 204 active+clean; 583 KiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:53.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-mon[41097]: osdmap e615: 8 total, 8 up, 8 in 2024-09-21T12:06:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-mon[30293]: pgmap v935: 204 pgs: 204 active+clean; 583 KiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-mon[30293]: osdmap e615: 8 total, 8 up, 8 in 2024-09-21T12:06:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:52 smithi193 ceph-mon[33333]: pgmap v935: 204 pgs: 204 active+clean; 583 KiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-21T12:06:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:52 smithi193 ceph-mon[33333]: osdmap e615: 8 total, 8 up, 8 in 2024-09-21T12:06:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:06:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[30293]: osdmap e616: 8 total, 8 up, 8 in 2024-09-21T12:06:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124"} : dispatch 2024-09-21T12:06:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[41097]: osdmap e616: 8 total, 8 up, 8 in 2024-09-21T12:06:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124"} : dispatch 2024-09-21T12:06:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:54.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:06:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:53 smithi193 ceph-mon[33333]: osdmap e616: 8 total, 8 up, 8 in 2024-09-21T12:06:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124"} : dispatch 2024-09-21T12:06:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:53 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:06:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[41097]: pgmap v938: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124"}]': finished 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[41097]: osdmap e617: 8 total, 8 up, 8 in 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[30293]: pgmap v938: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124"}]': finished 2024-09-21T12:06:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:54 smithi107 ceph-mon[30293]: osdmap e617: 8 total, 8 up, 8 in 2024-09-21T12:06:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:54 smithi193 ceph-mon[33333]: pgmap v938: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:06:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:54 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:06:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-124"}]': finished 2024-09-21T12:06:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:54 smithi193 ceph-mon[33333]: osdmap e617: 8 total, 8 up, 8 in 2024-09-21T12:06:56.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:55 smithi107 ceph-mon[30293]: osdmap e618: 8 total, 8 up, 8 in 2024-09-21T12:06:56.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:55 smithi107 ceph-mon[41097]: osdmap e618: 8 total, 8 up, 8 in 2024-09-21T12:06:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:55 smithi193 ceph-mon[33333]: osdmap e618: 8 total, 8 up, 8 in 2024-09-21T12:06:57.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:56 smithi107 ceph-mon[30293]: pgmap v941: 172 pgs: 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:56 smithi107 ceph-mon[30293]: osdmap e619: 8 total, 8 up, 8 in 2024-09-21T12:06:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:56 smithi107 ceph-mon[41097]: pgmap v941: 172 pgs: 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:56 smithi107 ceph-mon[41097]: osdmap e619: 8 total, 8 up, 8 in 2024-09-21T12:06:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:56 smithi193 ceph-mon[33333]: pgmap v941: 172 pgs: 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:06:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:56 smithi193 ceph-mon[33333]: osdmap e619: 8 total, 8 up, 8 in 2024-09-21T12:06:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-126","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:06:58.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[30293]: osdmap e620: 8 total, 8 up, 8 in 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[30293]: osdmap e621: 8 total, 8 up, 8 in 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-126"} : dispatch 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[41097]: osdmap e620: 8 total, 8 up, 8 in 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[41097]: osdmap e621: 8 total, 8 up, 8 in 2024-09-21T12:06:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-126"} : dispatch 2024-09-21T12:06:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:06:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:57 smithi193 ceph-mon[33333]: osdmap e620: 8 total, 8 up, 8 in 2024-09-21T12:06:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:06:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:06:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:57 smithi193 ceph-mon[33333]: osdmap e621: 8 total, 8 up, 8 in 2024-09-21T12:06:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-126"} : dispatch 2024-09-21T12:06:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[30293]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-126"}]': finished 2024-09-21T12:06:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[30293]: osdmap e622: 8 total, 8 up, 8 in 2024-09-21T12:06:59.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-126", "mode": "writeback"} : dispatch 2024-09-21T12:06:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[41097]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-126"}]': finished 2024-09-21T12:06:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[41097]: osdmap e622: 8 total, 8 up, 8 in 2024-09-21T12:06:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-126", "mode": "writeback"} : dispatch 2024-09-21T12:06:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:58 smithi193 ceph-mon[33333]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:06:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-126"}]': finished 2024-09-21T12:06:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:58 smithi193 ceph-mon[33333]: osdmap e622: 8 total, 8 up, 8 in 2024-09-21T12:06:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-126", "mode": "writeback"} : dispatch 2024-09-21T12:06:59.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:06:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:00.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-126", "mode": "writeback"}]': finished 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[30293]: osdmap e623: 8 total, 8 up, 8 in 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-126", "mode": "writeback"}]': finished 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[41097]: osdmap e623: 8 total, 8 up, 8 in 2024-09-21T12:07:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:06:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:59 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:59 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-126", "mode": "writeback"}]': finished 2024-09-21T12:07:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:59 smithi193 ceph-mon[33333]: osdmap e623: 8 total, 8 up, 8 in 2024-09-21T12:07:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:06:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:01.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[30293]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[30293]: osdmap e624: 8 total, 8 up, 8 in 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126"} : dispatch 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[41097]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[41097]: osdmap e624: 8 total, 8 up, 8 in 2024-09-21T12:07:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126"} : dispatch 2024-09-21T12:07:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:00 smithi193 ceph-mon[33333]: pgmap v948: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:07:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:00 smithi193 ceph-mon[33333]: osdmap e624: 8 total, 8 up, 8 in 2024-09-21T12:07:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126"} : dispatch 2024-09-21T12:07:02.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:01 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:02.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126"}]': finished 2024-09-21T12:07:02.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:01 smithi107 ceph-mon[30293]: osdmap e625: 8 total, 8 up, 8 in 2024-09-21T12:07:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:01 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126"}]': finished 2024-09-21T12:07:02.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:01 smithi107 ceph-mon[41097]: osdmap e625: 8 total, 8 up, 8 in 2024-09-21T12:07:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:01 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-126"}]': finished 2024-09-21T12:07:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:01 smithi193 ceph-mon[33333]: osdmap e625: 8 total, 8 up, 8 in 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9086 ms) 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6033 ms) 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:07:02.431 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5063 ms) 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (26106 ms) 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14068 ms) 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12086 ms) 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 00000012 2024-09-21T12:07:02.432 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12065 ms) 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = squid 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6029 ms) 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (554528 ms total) 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1010 ms) 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11017 ms) 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18084 ms) 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-21T12:07:02.433 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30113 ms total) 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7029 ms) 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8074 ms) 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: waiting for scrub... 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: done waiting 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24097 ms) 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9249 ms) 2024-09-21T12:07:02.434 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-21T12:07:02.435 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8045 ms) 2024-09-21T12:07:02.435 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-21T12:07:02.435 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8072 ms) 2024-09-21T12:07:02.435 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-21T12:07:02.435 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10071 ms) 2024-09-21T12:07:02.435 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-21T12:07:02.648 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:07:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:07:02.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:02 smithi193 ceph-mon[33333]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:02.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:02 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:02 smithi193 ceph-mon[33333]: osdmap e626: 8 total, 8 up, 8 in 2024-09-21T12:07:03.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-mon[30293]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:03.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:03.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-mon[30293]: osdmap e626: 8 total, 8 up, 8 in 2024-09-21T12:07:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-mon[41097]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:02 smithi107 ceph-mon[41097]: osdmap e626: 8 total, 8 up, 8 in 2024-09-21T12:07:04.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:04 smithi107 ceph-mon[30293]: osdmap e627: 8 total, 8 up, 8 in 2024-09-21T12:07:04.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:04.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:04 smithi107 ceph-mon[30293]: pgmap v954: 204 pgs: 3 creating+peering, 29 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:04 smithi107 ceph-mon[41097]: osdmap e627: 8 total, 8 up, 8 in 2024-09-21T12:07:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:04.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:04 smithi107 ceph-mon[41097]: pgmap v954: 204 pgs: 3 creating+peering, 29 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:04 smithi193 ceph-mon[33333]: osdmap e627: 8 total, 8 up, 8 in 2024-09-21T12:07:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-128","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:04.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:04 smithi193 ceph-mon[33333]: pgmap v954: 204 pgs: 3 creating+peering, 29 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:05.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:05 smithi107 ceph-mon[30293]: osdmap e628: 8 total, 8 up, 8 in 2024-09-21T12:07:05.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:05 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:05 smithi107 ceph-mon[41097]: osdmap e628: 8 total, 8 up, 8 in 2024-09-21T12:07:05.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:05 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:05 smithi193 ceph-mon[33333]: osdmap e628: 8 total, 8 up, 8 in 2024-09-21T12:07:05.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:05 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:06.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: osdmap e629: 8 total, 8 up, 8 in 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-128"} : dispatch 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: pgmap v957: 204 pgs: 19 creating+peering, 13 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-128"}]': finished 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: osdmap e630: 8 total, 8 up, 8 in 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-128", "mode": "writeback"} : dispatch 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: osdmap e629: 8 total, 8 up, 8 in 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-128"} : dispatch 2024-09-21T12:07:06.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: pgmap v957: 204 pgs: 19 creating+peering, 13 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-128"}]': finished 2024-09-21T12:07:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: osdmap e630: 8 total, 8 up, 8 in 2024-09-21T12:07:06.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-128", "mode": "writeback"} : dispatch 2024-09-21T12:07:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: osdmap e629: 8 total, 8 up, 8 in 2024-09-21T12:07:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-128"} : dispatch 2024-09-21T12:07:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: pgmap v957: 204 pgs: 19 creating+peering, 13 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-128"}]': finished 2024-09-21T12:07:06.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: osdmap e630: 8 total, 8 up, 8 in 2024-09-21T12:07:06.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-128", "mode": "writeback"} : dispatch 2024-09-21T12:07:07.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:07 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:07.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-128", "mode": "writeback"}]': finished 2024-09-21T12:07:07.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:07 smithi107 ceph-mon[41097]: osdmap e631: 8 total, 8 up, 8 in 2024-09-21T12:07:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:07 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-128", "mode": "writeback"}]': finished 2024-09-21T12:07:07.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:07 smithi107 ceph-mon[30293]: osdmap e631: 8 total, 8 up, 8 in 2024-09-21T12:07:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:07 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-128", "mode": "writeback"}]': finished 2024-09-21T12:07:07.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:07 smithi193 ceph-mon[33333]: osdmap e631: 8 total, 8 up, 8 in 2024-09-21T12:07:08.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:08 smithi107 ceph-mon[41097]: pgmap v960: 204 pgs: 19 creating+peering, 13 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:08.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:08 smithi107 ceph-mon[30293]: pgmap v960: 204 pgs: 19 creating+peering, 13 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:08.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:08 smithi193 ceph-mon[33333]: pgmap v960: 204 pgs: 19 creating+peering, 13 unknown, 172 active+clean; 583 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:08.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:09.460 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:07:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:09.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:09 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:09.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:09 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:09.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:09 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:10.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:10 smithi107 ceph-mon[41097]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 402 B/s wr, 1 op/s 2024-09-21T12:07:10.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:10 smithi107 ceph-mon[30293]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 402 B/s wr, 1 op/s 2024-09-21T12:07:10.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:10 smithi193 ceph-mon[33333]: pgmap v961: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 402 B/s wr, 1 op/s 2024-09-21T12:07:12.571 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:07:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:07:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-mon[30293]: pgmap v962: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1010 B/s rd, 336 B/s wr, 1 op/s 2024-09-21T12:07:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-mon[41097]: pgmap v962: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1010 B/s rd, 336 B/s wr, 1 op/s 2024-09-21T12:07:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:12 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:12 smithi193 ceph-mon[33333]: pgmap v962: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1010 B/s rd, 336 B/s wr, 1 op/s 2024-09-21T12:07:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:12 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:12 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:14.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:14.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:13 smithi107 ceph-mon[30293]: osdmap e632: 8 total, 8 up, 8 in 2024-09-21T12:07:14.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128"} : dispatch 2024-09-21T12:07:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:13 smithi107 ceph-mon[41097]: osdmap e632: 8 total, 8 up, 8 in 2024-09-21T12:07:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128"} : dispatch 2024-09-21T12:07:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:13 smithi193 ceph-mon[33333]: osdmap e632: 8 total, 8 up, 8 in 2024-09-21T12:07:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128"} : dispatch 2024-09-21T12:07:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[30293]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 289 B/s wr, 1 op/s 2024-09-21T12:07:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128"}]': finished 2024-09-21T12:07:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[30293]: osdmap e633: 8 total, 8 up, 8 in 2024-09-21T12:07:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[41097]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 289 B/s wr, 1 op/s 2024-09-21T12:07:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128"}]': finished 2024-09-21T12:07:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:14 smithi107 ceph-mon[41097]: osdmap e633: 8 total, 8 up, 8 in 2024-09-21T12:07:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:14 smithi193 ceph-mon[33333]: pgmap v964: 204 pgs: 204 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 289 B/s wr, 1 op/s 2024-09-21T12:07:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:14 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-128"}]': finished 2024-09-21T12:07:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:14 smithi193 ceph-mon[33333]: osdmap e633: 8 total, 8 up, 8 in 2024-09-21T12:07:16.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:15 smithi107 ceph-mon[30293]: osdmap e634: 8 total, 8 up, 8 in 2024-09-21T12:07:16.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:15 smithi107 ceph-mon[30293]: osdmap e635: 8 total, 8 up, 8 in 2024-09-21T12:07:16.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:15 smithi107 ceph-mon[41097]: osdmap e634: 8 total, 8 up, 8 in 2024-09-21T12:07:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:15 smithi107 ceph-mon[41097]: osdmap e635: 8 total, 8 up, 8 in 2024-09-21T12:07:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:15 smithi193 ceph-mon[33333]: osdmap e634: 8 total, 8 up, 8 in 2024-09-21T12:07:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:15 smithi193 ceph-mon[33333]: osdmap e635: 8 total, 8 up, 8 in 2024-09-21T12:07:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-130","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[30293]: pgmap v967: 172 pgs: 172 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T12:07:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[30293]: osdmap e636: 8 total, 8 up, 8 in 2024-09-21T12:07:17.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[41097]: pgmap v967: 172 pgs: 172 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T12:07:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[41097]: osdmap e636: 8 total, 8 up, 8 in 2024-09-21T12:07:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:16 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:16 smithi193 ceph-mon[33333]: pgmap v967: 172 pgs: 172 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T12:07:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:16 smithi193 ceph-mon[33333]: osdmap e636: 8 total, 8 up, 8 in 2024-09-21T12:07:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:16 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[30293]: pgmap v970: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[30293]: osdmap e637: 8 total, 8 up, 8 in 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-130"} : dispatch 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[41097]: pgmap v970: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[41097]: osdmap e637: 8 total, 8 up, 8 in 2024-09-21T12:07:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-130"} : dispatch 2024-09-21T12:07:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:18 smithi193 ceph-mon[33333]: pgmap v970: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:18 smithi193 ceph-mon[33333]: osdmap e637: 8 total, 8 up, 8 in 2024-09-21T12:07:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-130"} : dispatch 2024-09-21T12:07:19.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:20.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-130"}]': finished 2024-09-21T12:07:20.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[30293]: osdmap e638: 8 total, 8 up, 8 in 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-130", "mode": "writeback"} : dispatch 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-130", "mode": "writeback"}]': finished 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-130"}]': finished 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[41097]: osdmap e638: 8 total, 8 up, 8 in 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-130", "mode": "writeback"} : dispatch 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:20.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-130", "mode": "writeback"}]': finished 2024-09-21T12:07:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-130"}]': finished 2024-09-21T12:07:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-mon[33333]: osdmap e638: 8 total, 8 up, 8 in 2024-09-21T12:07:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-130", "mode": "writeback"} : dispatch 2024-09-21T12:07:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:20.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:20.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-130", "mode": "writeback"}]': finished 2024-09-21T12:07:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:20 smithi107 ceph-mon[30293]: pgmap v973: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:20 smithi107 ceph-mon[30293]: osdmap e639: 8 total, 8 up, 8 in 2024-09-21T12:07:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:20 smithi107 ceph-mon[41097]: pgmap v973: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:20 smithi107 ceph-mon[41097]: osdmap e639: 8 total, 8 up, 8 in 2024-09-21T12:07:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:20 smithi193 ceph-mon[33333]: pgmap v973: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:20 smithi193 ceph-mon[33333]: osdmap e639: 8 total, 8 up, 8 in 2024-09-21T12:07:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:22.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:21 smithi107 ceph-mon[30293]: osdmap e640: 8 total, 8 up, 8 in 2024-09-21T12:07:22.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130"} : dispatch 2024-09-21T12:07:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:21 smithi107 ceph-mon[41097]: osdmap e640: 8 total, 8 up, 8 in 2024-09-21T12:07:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130"} : dispatch 2024-09-21T12:07:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:21 smithi193 ceph-mon[33333]: osdmap e640: 8 total, 8 up, 8 in 2024-09-21T12:07:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130"} : dispatch 2024-09-21T12:07:22.638 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:07:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:07:22.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[30293]: pgmap v976: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:22.641 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:22.641 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130"}]': finished 2024-09-21T12:07:22.641 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[30293]: osdmap e641: 8 total, 8 up, 8 in 2024-09-21T12:07:23.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[41097]: pgmap v976: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130"}]': finished 2024-09-21T12:07:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:22 smithi107 ceph-mon[41097]: osdmap e641: 8 total, 8 up, 8 in 2024-09-21T12:07:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:22 smithi193 ceph-mon[33333]: pgmap v976: 204 pgs: 204 active+clean; 583 KiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:22 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-130"}]': finished 2024-09-21T12:07:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:22 smithi193 ceph-mon[33333]: osdmap e641: 8 total, 8 up, 8 in 2024-09-21T12:07:24.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[41097]: osdmap e642: 8 total, 8 up, 8 in 2024-09-21T12:07:24.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[41097]: osdmap e643: 8 total, 8 up, 8 in 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[30293]: osdmap e642: 8 total, 8 up, 8 in 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[30293]: osdmap e643: 8 total, 8 up, 8 in 2024-09-21T12:07:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:23 smithi193 ceph-mon[33333]: osdmap e642: 8 total, 8 up, 8 in 2024-09-21T12:07:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:23 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:07:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:23 smithi193 ceph-mon[33333]: osdmap e643: 8 total, 8 up, 8 in 2024-09-21T12:07:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-132","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[30293]: pgmap v979: 172 pgs: 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[30293]: osdmap e644: 8 total, 8 up, 8 in 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[41097]: pgmap v979: 172 pgs: 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[41097]: osdmap e644: 8 total, 8 up, 8 in 2024-09-21T12:07:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:24 smithi193 ceph-mon[33333]: pgmap v979: 172 pgs: 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:24 smithi193 ceph-mon[33333]: osdmap e644: 8 total, 8 up, 8 in 2024-09-21T12:07:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:27.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[30293]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[30293]: osdmap e645: 8 total, 8 up, 8 in 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[41097]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[41097]: osdmap e645: 8 total, 8 up, 8 in 2024-09-21T12:07:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:26 smithi193 ceph-mon[33333]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:26 smithi193 ceph-mon[33333]: osdmap e645: 8 total, 8 up, 8 in 2024-09-21T12:07:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:28.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[41097]: osdmap e646: 8 total, 8 up, 8 in 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-132", "mode": "writeback"} : dispatch 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-132", "mode": "writeback"}]': finished 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[30293]: osdmap e646: 8 total, 8 up, 8 in 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-132", "mode": "writeback"} : dispatch 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-132", "mode": "writeback"}]': finished 2024-09-21T12:07:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:27 smithi193 ceph-mon[33333]: osdmap e646: 8 total, 8 up, 8 in 2024-09-21T12:07:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-132", "mode": "writeback"} : dispatch 2024-09-21T12:07:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:27 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-132", "mode": "writeback"}]': finished 2024-09-21T12:07:29.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:28 smithi107 ceph-mon[30293]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:29.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:28 smithi107 ceph-mon[30293]: osdmap e647: 8 total, 8 up, 8 in 2024-09-21T12:07:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:28 smithi107 ceph-mon[41097]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:28 smithi107 ceph-mon[41097]: osdmap e647: 8 total, 8 up, 8 in 2024-09-21T12:07:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:28 smithi193 ceph-mon[33333]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:28 smithi193 ceph-mon[33333]: osdmap e647: 8 total, 8 up, 8 in 2024-09-21T12:07:29.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:07:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:30.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:29 smithi107 ceph-mon[30293]: osdmap e648: 8 total, 8 up, 8 in 2024-09-21T12:07:30.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:29 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:30.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:29 smithi107 ceph-mon[41097]: osdmap e648: 8 total, 8 up, 8 in 2024-09-21T12:07:30.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:29 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:29 smithi193 ceph-mon[33333]: osdmap e648: 8 total, 8 up, 8 in 2024-09-21T12:07:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:29 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: osdmap e649: 8 total, 8 up, 8 in 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: osdmap e649: 8 total, 8 up, 8 in 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:07:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:30 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: pgmap v988: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: osdmap e649: 8 total, 8 up, 8 in 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:07:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:30 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:07:32.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:31 smithi107 ceph-mon[41097]: osdmap e650: 8 total, 8 up, 8 in 2024-09-21T12:07:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:31 smithi107 ceph-mon[30293]: osdmap e650: 8 total, 8 up, 8 in 2024-09-21T12:07:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:31 smithi193 ceph-mon[33333]: osdmap e650: 8 total, 8 up, 8 in 2024-09-21T12:07:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:32.671 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:07:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:07:32.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:32 smithi193 ceph-mon[33333]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:32.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:32.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:32 smithi193 ceph-mon[33333]: osdmap e651: 8 total, 8 up, 8 in 2024-09-21T12:07:33.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-mon[41097]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-mon[41097]: osdmap e651: 8 total, 8 up, 8 in 2024-09-21T12:07:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-mon[30293]: pgmap v991: 204 pgs: 204 active+clean; 583 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:07:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:32 smithi107 ceph-mon[30293]: osdmap e651: 8 total, 8 up, 8 in 2024-09-21T12:07:34.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:33 smithi107 ceph-mon[41097]: osdmap e652: 8 total, 8 up, 8 in 2024-09-21T12:07:34.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:34.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:33 smithi107 ceph-mon[30293]: osdmap e652: 8 total, 8 up, 8 in 2024-09-21T12:07:34.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:34.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:33 smithi193 ceph-mon[33333]: osdmap e652: 8 total, 8 up, 8 in 2024-09-21T12:07:34.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[30293]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:07:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[30293]: osdmap e653: 8 total, 8 up, 8 in 2024-09-21T12:07:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[41097]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:07:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[41097]: osdmap e653: 8 total, 8 up, 8 in 2024-09-21T12:07:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:34 smithi193 ceph-mon[33333]: pgmap v994: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:07:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:34 smithi193 ceph-mon[33333]: osdmap e653: 8 total, 8 up, 8 in 2024-09-21T12:07:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132"} : dispatch 2024-09-21T12:07:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:35 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:35 smithi107 ceph-mon[30293]: osdmap e654: 8 total, 8 up, 8 in 2024-09-21T12:07:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:35 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:35 smithi107 ceph-mon[41097]: osdmap e654: 8 total, 8 up, 8 in 2024-09-21T12:07:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:35 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-132"}]': finished 2024-09-21T12:07:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:35 smithi193 ceph-mon[33333]: osdmap e654: 8 total, 8 up, 8 in 2024-09-21T12:07:37.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:36 smithi107 ceph-mon[30293]: pgmap v997: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:07:37.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:36 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:37.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:36 smithi107 ceph-mon[30293]: osdmap e655: 8 total, 8 up, 8 in 2024-09-21T12:07:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:36 smithi107 ceph-mon[41097]: pgmap v997: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:07:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:36 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:36 smithi107 ceph-mon[41097]: osdmap e655: 8 total, 8 up, 8 in 2024-09-21T12:07:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:36 smithi193 ceph-mon[33333]: pgmap v997: 204 pgs: 204 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-21T12:07:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:36 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:36 smithi193 ceph-mon[33333]: osdmap e655: 8 total, 8 up, 8 in 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[30293]: osdmap e656: 8 total, 8 up, 8 in 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[30293]: osdmap e657: 8 total, 8 up, 8 in 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[41097]: osdmap e656: 8 total, 8 up, 8 in 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[41097]: osdmap e657: 8 total, 8 up, 8 in 2024-09-21T12:07:38.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:37 smithi193 ceph-mon[33333]: osdmap e656: 8 total, 8 up, 8 in 2024-09-21T12:07:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-134","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:37 smithi193 ceph-mon[33333]: osdmap e657: 8 total, 8 up, 8 in 2024-09-21T12:07:38.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:39.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[30293]: pgmap v1001: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[30293]: osdmap e658: 8 total, 8 up, 8 in 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-134"} : dispatch 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[41097]: pgmap v1001: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[41097]: osdmap e658: 8 total, 8 up, 8 in 2024-09-21T12:07:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-134"} : dispatch 2024-09-21T12:07:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:38 smithi193 ceph-mon[33333]: pgmap v1001: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:38 smithi193 ceph-mon[33333]: osdmap e658: 8 total, 8 up, 8 in 2024-09-21T12:07:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-134"} : dispatch 2024-09-21T12:07:39.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:07:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:40.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:40.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-134"}]': finished 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[30293]: osdmap e659: 8 total, 8 up, 8 in 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-134", "mode": "writeback"} : dispatch 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[30293]: pgmap v1004: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-134"}]': finished 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[41097]: osdmap e659: 8 total, 8 up, 8 in 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-134", "mode": "writeback"} : dispatch 2024-09-21T12:07:40.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:40 smithi107 ceph-mon[41097]: pgmap v1004: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:07:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:40 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-134"}]': finished 2024-09-21T12:07:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:40 smithi193 ceph-mon[33333]: osdmap e659: 8 total, 8 up, 8 in 2024-09-21T12:07:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-134", "mode": "writeback"} : dispatch 2024-09-21T12:07:40.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:40 smithi193 ceph-mon[33333]: pgmap v1004: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-134", "mode": "writeback"}]': finished 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[30293]: osdmap e660: 8 total, 8 up, 8 in 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-134", "mode": "writeback"}]': finished 2024-09-21T12:07:41.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[41097]: osdmap e660: 8 total, 8 up, 8 in 2024-09-21T12:07:41.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:41 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:41.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-134", "mode": "writeback"}]': finished 2024-09-21T12:07:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:41 smithi193 ceph-mon[33333]: osdmap e660: 8 total, 8 up, 8 in 2024-09-21T12:07:41.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:07:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: osdmap e661: 8 total, 8 up, 8 in 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134"} : dispatch 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: pgmap v1007: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:42.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134"}]': finished 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[30293]: osdmap e662: 8 total, 8 up, 8 in 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: osdmap e661: 8 total, 8 up, 8 in 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134"} : dispatch 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: pgmap v1007: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134"}]': finished 2024-09-21T12:07:42.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:42 smithi107 ceph-mon[41097]: osdmap e662: 8 total, 8 up, 8 in 2024-09-21T12:07:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: osdmap e661: 8 total, 8 up, 8 in 2024-09-21T12:07:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134"} : dispatch 2024-09-21T12:07:42.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: pgmap v1007: 204 pgs: 204 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:07:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-134"}]': finished 2024-09-21T12:07:42.801 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:42 smithi193 ceph-mon[33333]: osdmap e662: 8 total, 8 up, 8 in 2024-09-21T12:07:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:44 smithi107 ceph-mon[30293]: osdmap e663: 8 total, 8 up, 8 in 2024-09-21T12:07:44.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:44 smithi107 ceph-mon[30293]: pgmap v1010: 172 pgs: 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T12:07:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:44 smithi107 ceph-mon[41097]: osdmap e663: 8 total, 8 up, 8 in 2024-09-21T12:07:44.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:44 smithi107 ceph-mon[41097]: pgmap v1010: 172 pgs: 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T12:07:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:44 smithi193 ceph-mon[33333]: osdmap e663: 8 total, 8 up, 8 in 2024-09-21T12:07:44.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:44 smithi193 ceph-mon[33333]: pgmap v1010: 172 pgs: 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T12:07:45.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:45 smithi107 ceph-mon[30293]: osdmap e664: 8 total, 8 up, 8 in 2024-09-21T12:07:45.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:45 smithi107 ceph-mon[41097]: osdmap e664: 8 total, 8 up, 8 in 2024-09-21T12:07:45.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:45 smithi193 ceph-mon[33333]: osdmap e664: 8 total, 8 up, 8 in 2024-09-21T12:07:45.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-136","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[30293]: osdmap e665: 8 total, 8 up, 8 in 2024-09-21T12:07:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:46.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[30293]: pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T12:07:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[41097]: osdmap e665: 8 total, 8 up, 8 in 2024-09-21T12:07:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:46.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:46 smithi107 ceph-mon[41097]: pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T12:07:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:46 smithi193 ceph-mon[33333]: osdmap e665: 8 total, 8 up, 8 in 2024-09-21T12:07:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:46.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:46 smithi193 ceph-mon[33333]: pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-21T12:07:47.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:47 smithi107 ceph-mon[30293]: osdmap e666: 8 total, 8 up, 8 in 2024-09-21T12:07:47.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-136"} : dispatch 2024-09-21T12:07:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:47 smithi107 ceph-mon[41097]: osdmap e666: 8 total, 8 up, 8 in 2024-09-21T12:07:47.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-136"} : dispatch 2024-09-21T12:07:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:47 smithi193 ceph-mon[33333]: osdmap e666: 8 total, 8 up, 8 in 2024-09-21T12:07:47.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-136"} : dispatch 2024-09-21T12:07:48.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-136"}]': finished 2024-09-21T12:07:48.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: osdmap e667: 8 total, 8 up, 8 in 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-136", "mode": "writeback"} : dispatch 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-136", "mode": "writeback"}]': finished 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[30293]: osdmap e668: 8 total, 8 up, 8 in 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-136"}]': finished 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: osdmap e667: 8 total, 8 up, 8 in 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-136", "mode": "writeback"} : dispatch 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-136", "mode": "writeback"}]': finished 2024-09-21T12:07:48.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:48 smithi107 ceph-mon[41097]: osdmap e668: 8 total, 8 up, 8 in 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-136"}]': finished 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: osdmap e667: 8 total, 8 up, 8 in 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-136", "mode": "writeback"} : dispatch 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 583 KiB data, 684 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-136", "mode": "writeback"}]': finished 2024-09-21T12:07:48.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:48 smithi193 ceph-mon[33333]: osdmap e668: 8 total, 8 up, 8 in 2024-09-21T12:07:49.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:07:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:49.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:49.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:49.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:50.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:50 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:50.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:50.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:50 smithi193 ceph-mon[33333]: osdmap e669: 8 total, 8 up, 8 in 2024-09-21T12:07:50.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136"} : dispatch 2024-09-21T12:07:50.802 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:50 smithi193 ceph-mon[33333]: pgmap v1019: 204 pgs: 204 active+clean; 4.4 MiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-21T12:07:51.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[30293]: osdmap e669: 8 total, 8 up, 8 in 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136"} : dispatch 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[30293]: pgmap v1019: 204 pgs: 204 active+clean; 4.4 MiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[41097]: osdmap e669: 8 total, 8 up, 8 in 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136"} : dispatch 2024-09-21T12:07:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:50 smithi107 ceph-mon[41097]: pgmap v1019: 204 pgs: 204 active+clean; 4.4 MiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-21T12:07:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:51 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136"}]': finished 2024-09-21T12:07:51.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:51 smithi193 ceph-mon[33333]: osdmap e670: 8 total, 8 up, 8 in 2024-09-21T12:07:52.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:51 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:52.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136"}]': finished 2024-09-21T12:07:52.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:51 smithi107 ceph-mon[30293]: osdmap e670: 8 total, 8 up, 8 in 2024-09-21T12:07:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:51 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:07:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-136"}]': finished 2024-09-21T12:07:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:51 smithi107 ceph-mon[41097]: osdmap e670: 8 total, 8 up, 8 in 2024-09-21T12:07:52.535 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:07:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:07:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:52 smithi193 ceph-mon[33333]: osdmap e671: 8 total, 8 up, 8 in 2024-09-21T12:07:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:52 smithi193 ceph-mon[33333]: pgmap v1022: 172 pgs: 172 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:52.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:52 smithi193 ceph-mon[33333]: osdmap e672: 8 total, 8 up, 8 in 2024-09-21T12:07:53.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-mon[30293]: osdmap e671: 8 total, 8 up, 8 in 2024-09-21T12:07:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-mon[30293]: pgmap v1022: 172 pgs: 172 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-mon[30293]: osdmap e672: 8 total, 8 up, 8 in 2024-09-21T12:07:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-mon[41097]: osdmap e671: 8 total, 8 up, 8 in 2024-09-21T12:07:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-mon[41097]: pgmap v1022: 172 pgs: 172 active+clean; 583 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:52 smithi107 ceph-mon[41097]: osdmap e672: 8 total, 8 up, 8 in 2024-09-21T12:07:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:53 smithi193 ceph-mon[33333]: osdmap e673: 8 total, 8 up, 8 in 2024-09-21T12:07:53.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:54.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:54.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:54.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[41097]: osdmap e673: 8 total, 8 up, 8 in 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-138","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[30293]: osdmap e673: 8 total, 8 up, 8 in 2024-09-21T12:07:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:07:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:54 smithi193 ceph-mon[33333]: pgmap v1025: 204 pgs: 2 creating+activating, 23 creating+peering, 5 unknown, 174 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:54 smithi193 ceph-mon[33333]: osdmap e674: 8 total, 8 up, 8 in 2024-09-21T12:07:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:07:55.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[30293]: pgmap v1025: 204 pgs: 2 creating+activating, 23 creating+peering, 5 unknown, 174 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:55.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[30293]: osdmap e674: 8 total, 8 up, 8 in 2024-09-21T12:07:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:07:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[41097]: pgmap v1025: 204 pgs: 2 creating+activating, 23 creating+peering, 5 unknown, 174 active+clean; 4.4 MiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:07:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:07:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[41097]: osdmap e674: 8 total, 8 up, 8 in 2024-09-21T12:07:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:07:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:07:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:56 smithi193 ceph-mon[33333]: osdmap e675: 8 total, 8 up, 8 in 2024-09-21T12:07:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:07:56.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:56 smithi193 ceph-mon[33333]: pgmap v1028: 204 pgs: 2 creating+activating, 23 creating+peering, 179 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-21T12:07:57.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:07:57.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[30293]: osdmap e675: 8 total, 8 up, 8 in 2024-09-21T12:07:57.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:07:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[30293]: pgmap v1028: 204 pgs: 2 creating+activating, 23 creating+peering, 179 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-21T12:07:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:07:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[41097]: osdmap e675: 8 total, 8 up, 8 in 2024-09-21T12:07:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:07:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:56 smithi107 ceph-mon[41097]: pgmap v1028: 204 pgs: 2 creating+activating, 23 creating+peering, 179 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-21T12:07:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:07:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:57 smithi193 ceph-mon[33333]: osdmap e676: 8 total, 8 up, 8 in 2024-09-21T12:07:57.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:07:58.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:07:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:57 smithi107 ceph-mon[30293]: osdmap e676: 8 total, 8 up, 8 in 2024-09-21T12:07:58.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:07:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:07:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:57 smithi107 ceph-mon[41097]: osdmap e676: 8 total, 8 up, 8 in 2024-09-21T12:07:58.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_type","val": "explicit_object"} : dispatch 2024-09-21T12:07:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:07:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:58 smithi193 ceph-mon[33333]: osdmap e677: 8 total, 8 up, 8 in 2024-09-21T12:07:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:58 smithi193 ceph-mon[33333]: pgmap v1031: 204 pgs: 2 creating+activating, 23 creating+peering, 179 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:07:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:58 smithi107 ceph-mon[30293]: osdmap e677: 8 total, 8 up, 8 in 2024-09-21T12:07:59.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:58 smithi107 ceph-mon[30293]: pgmap v1031: 204 pgs: 2 creating+activating, 23 creating+peering, 179 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-21T12:07:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:58 smithi107 ceph-mon[41097]: osdmap e677: 8 total, 8 up, 8 in 2024-09-21T12:07:59.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:58 smithi107 ceph-mon[41097]: pgmap v1031: 204 pgs: 2 creating+activating, 23 creating+peering, 179 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:07:59.543 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:07:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:07:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:07:59.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:07:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138"} : dispatch 2024-09-21T12:08:00.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:00.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:07:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138"} : dispatch 2024-09-21T12:08:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:00.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:07:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138"} : dispatch 2024-09-21T12:08:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:00 smithi193 ceph-mon[33333]: pgmap v1032: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-21T12:08:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138"}]': finished 2024-09-21T12:08:00.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:00 smithi193 ceph-mon[33333]: osdmap e678: 8 total, 8 up, 8 in 2024-09-21T12:08:01.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:01.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[41097]: pgmap v1032: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-21T12:08:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138"}]': finished 2024-09-21T12:08:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[41097]: osdmap e678: 8 total, 8 up, 8 in 2024-09-21T12:08:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[30293]: pgmap v1032: 204 pgs: 204 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1020 B/s rd, 0 op/s 2024-09-21T12:08:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-138"}]': finished 2024-09-21T12:08:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:00 smithi107 ceph-mon[30293]: osdmap e678: 8 total, 8 up, 8 in 2024-09-21T12:08:02.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:01 smithi107 ceph-mon[41097]: osdmap e679: 8 total, 8 up, 8 in 2024-09-21T12:08:02.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:01 smithi107 ceph-mon[30293]: osdmap e679: 8 total, 8 up, 8 in 2024-09-21T12:08:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:01 smithi193 ceph-mon[33333]: osdmap e679: 8 total, 8 up, 8 in 2024-09-21T12:08:02.564 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:08:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:08:02.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:02 smithi193 ceph-mon[33333]: pgmap v1035: 172 pgs: 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-09-21T12:08:02.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:02 smithi193 ceph-mon[33333]: osdmap e680: 8 total, 8 up, 8 in 2024-09-21T12:08:02.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:03.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-mon[41097]: pgmap v1035: 172 pgs: 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-09-21T12:08:03.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-mon[41097]: osdmap e680: 8 total, 8 up, 8 in 2024-09-21T12:08:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-mon[30293]: pgmap v1035: 172 pgs: 172 active+clean; 4.4 MiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-09-21T12:08:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-mon[30293]: osdmap e680: 8 total, 8 up, 8 in 2024-09-21T12:08:03.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-140","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:04.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[41097]: osdmap e681: 8 total, 8 up, 8 in 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[30293]: osdmap e681: 8 total, 8 up, 8 in 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:04.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:03 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:03 smithi193 ceph-mon[33333]: osdmap e681: 8 total, 8 up, 8 in 2024-09-21T12:08:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:04.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:03 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:05.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[41097]: pgmap v1038: 204 pgs: 11 unknown, 193 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[41097]: osdmap e682: 8 total, 8 up, 8 in 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_count","val": "3"}]': finished 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[41097]: osdmap e683: 8 total, 8 up, 8 in 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[30293]: pgmap v1038: 204 pgs: 11 unknown, 193 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[30293]: osdmap e682: 8 total, 8 up, 8 in 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_count","val": "3"}]': finished 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[30293]: osdmap e683: 8 total, 8 up, 8 in 2024-09-21T12:08:05.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-21T12:08:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:04 smithi193 ceph-mon[33333]: pgmap v1038: 204 pgs: 11 unknown, 193 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:04 smithi193 ceph-mon[33333]: osdmap e682: 8 total, 8 up, 8 in 2024-09-21T12:08:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_count","val": "3"} : dispatch 2024-09-21T12:08:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_count","val": "3"}]': finished 2024-09-21T12:08:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:04 smithi193 ceph-mon[33333]: osdmap e683: 8 total, 8 up, 8 in 2024-09-21T12:08:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_period","val": "3"} : dispatch 2024-09-21T12:08:07.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[41097]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:07.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_period","val": "3"}]': finished 2024-09-21T12:08:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[41097]: osdmap e684: 8 total, 8 up, 8 in 2024-09-21T12:08:07.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:08:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[30293]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_period","val": "3"}]': finished 2024-09-21T12:08:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[30293]: osdmap e684: 8 total, 8 up, 8 in 2024-09-21T12:08:07.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:06 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:08:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:06 smithi193 ceph-mon[33333]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_period","val": "3"}]': finished 2024-09-21T12:08:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:06 smithi193 ceph-mon[33333]: osdmap e684: 8 total, 8 up, 8 in 2024-09-21T12:08:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:06 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[41097]: osdmap e685: 8 total, 8 up, 8 in 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[41097]: osdmap e686: 8 total, 8 up, 8 in 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[30293]: osdmap e685: 8 total, 8 up, 8 in 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-21T12:08:08.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-21T12:08:08.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:07 smithi107 ceph-mon[30293]: osdmap e686: 8 total, 8 up, 8 in 2024-09-21T12:08:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:08:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:07 smithi193 ceph-mon[33333]: osdmap e685: 8 total, 8 up, 8 in 2024-09-21T12:08:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_fpp","val": ".01"} : dispatch 2024-09-21T12:08:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:07 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-21T12:08:08.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:07 smithi193 ceph-mon[33333]: osdmap e686: 8 total, 8 up, 8 in 2024-09-21T12:08:09.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:08 smithi107 ceph-mon[41097]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:09.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:08 smithi107 ceph-mon[30293]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:08 smithi193 ceph-mon[33333]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:09.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:08:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:08:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:10 smithi107 ceph-mon[41097]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-21T12:08:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:10 smithi107 ceph-mon[30293]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-21T12:08:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:10 smithi193 ceph-mon[33333]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-21T12:08:12.591 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:08:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:08:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:08:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:12 smithi107 ceph-mon[30293]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-09-21T12:08:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:12 smithi107 ceph-mon[41097]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-09-21T12:08:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:12 smithi193 ceph-mon[33333]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-09-21T12:08:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:14 smithi107 ceph-mon[30293]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-09-21T12:08:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:14 smithi107 ceph-mon[41097]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-09-21T12:08:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:14 smithi193 ceph-mon[33333]: pgmap v1048: 204 pgs: 204 active+clean; 4.4 MiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-09-21T12:08:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:16 smithi107 ceph-mon[30293]: pgmap v1049: 204 pgs: 204 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-21T12:08:17.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:16 smithi107 ceph-mon[41097]: pgmap v1049: 204 pgs: 204 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-21T12:08:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:16 smithi193 ceph-mon[33333]: pgmap v1049: 204 pgs: 204 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-21T12:08:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:18 smithi107 ceph-mon[30293]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-21T12:08:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:18 smithi107 ceph-mon[41097]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-21T12:08:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:18 smithi193 ceph-mon[33333]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-09-21T12:08:19.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:08:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:08:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[30293]: pgmap v1051: 204 pgs: 204 active+clean; 4.5 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 7.6 KiB/s wr, 2 op/s 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140"} : dispatch 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[41097]: pgmap v1051: 204 pgs: 204 active+clean; 4.5 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 7.6 KiB/s wr, 2 op/s 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140"} : dispatch 2024-09-21T12:08:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:20 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:20 smithi193 ceph-mon[33333]: pgmap v1051: 204 pgs: 204 active+clean; 4.5 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 7.6 KiB/s wr, 2 op/s 2024-09-21T12:08:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140"} : dispatch 2024-09-21T12:08:22.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140"}]': finished 2024-09-21T12:08:22.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:21 smithi107 ceph-mon[30293]: osdmap e687: 8 total, 8 up, 8 in 2024-09-21T12:08:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140"}]': finished 2024-09-21T12:08:22.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:21 smithi107 ceph-mon[41097]: osdmap e687: 8 total, 8 up, 8 in 2024-09-21T12:08:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-140"}]': finished 2024-09-21T12:08:22.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:21 smithi193 ceph-mon[33333]: osdmap e687: 8 total, 8 up, 8 in 2024-09-21T12:08:22.633 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:08:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:08:23.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-mon[30293]: pgmap v1053: 204 pgs: 204 active+clean; 4.5 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-21T12:08:23.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:23.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-mon[30293]: osdmap e688: 8 total, 8 up, 8 in 2024-09-21T12:08:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-mon[41097]: pgmap v1053: 204 pgs: 204 active+clean; 4.5 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-21T12:08:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:22 smithi107 ceph-mon[41097]: osdmap e688: 8 total, 8 up, 8 in 2024-09-21T12:08:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:22 smithi193 ceph-mon[33333]: pgmap v1053: 204 pgs: 204 active+clean; 4.5 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-21T12:08:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:22 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:22 smithi193 ceph-mon[33333]: osdmap e688: 8 total, 8 up, 8 in 2024-09-21T12:08:24.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[30293]: osdmap e689: 8 total, 8 up, 8 in 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[30293]: osdmap e690: 8 total, 8 up, 8 in 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[41097]: osdmap e689: 8 total, 8 up, 8 in 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:23 smithi107 ceph-mon[41097]: osdmap e690: 8 total, 8 up, 8 in 2024-09-21T12:08:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:23 smithi193 ceph-mon[33333]: osdmap e689: 8 total, 8 up, 8 in 2024-09-21T12:08:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-142","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:23 smithi193 ceph-mon[33333]: osdmap e690: 8 total, 8 up, 8 in 2024-09-21T12:08:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[30293]: pgmap v1056: 204 pgs: 25 creating+peering, 7 unknown, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[30293]: osdmap e691: 8 total, 8 up, 8 in 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[41097]: pgmap v1056: 204 pgs: 25 creating+peering, 7 unknown, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[41097]: osdmap e691: 8 total, 8 up, 8 in 2024-09-21T12:08:25.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:24 smithi193 ceph-mon[33333]: pgmap v1056: 204 pgs: 25 creating+peering, 7 unknown, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:24 smithi193 ceph-mon[33333]: osdmap e691: 8 total, 8 up, 8 in 2024-09-21T12:08:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:27.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[41097]: pgmap v1059: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-142"}]': finished 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[41097]: osdmap e692: 8 total, 8 up, 8 in 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-142", "mode": "writeback"} : dispatch 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[30293]: pgmap v1059: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-142"}]': finished 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[30293]: osdmap e692: 8 total, 8 up, 8 in 2024-09-21T12:08:27.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-142", "mode": "writeback"} : dispatch 2024-09-21T12:08:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:26 smithi193 ceph-mon[33333]: pgmap v1059: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-142"}]': finished 2024-09-21T12:08:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:26 smithi193 ceph-mon[33333]: osdmap e692: 8 total, 8 up, 8 in 2024-09-21T12:08:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-142", "mode": "writeback"} : dispatch 2024-09-21T12:08:28.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:28.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-142", "mode": "writeback"}]': finished 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: osdmap e693: 8 total, 8 up, 8 in 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: osdmap e694: 8 total, 8 up, 8 in 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-142", "mode": "writeback"}]': finished 2024-09-21T12:08:28.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: osdmap e693: 8 total, 8 up, 8 in 2024-09-21T12:08:28.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:08:28.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:28.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:08:28.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: osdmap e694: 8 total, 8 up, 8 in 2024-09-21T12:08:28.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-142", "mode": "writeback"}]': finished 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: osdmap e693: 8 total, 8 up, 8 in 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: osdmap e694: 8 total, 8 up, 8 in 2024-09-21T12:08:28.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:08:29.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[30293]: pgmap v1062: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:29.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:08:29.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[30293]: osdmap e695: 8 total, 8 up, 8 in 2024-09-21T12:08:29.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:08:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[41097]: pgmap v1062: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:08:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[41097]: osdmap e695: 8 total, 8 up, 8 in 2024-09-21T12:08:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:08:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:28 smithi193 ceph-mon[33333]: pgmap v1062: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 695 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:08:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:28 smithi193 ceph-mon[33333]: osdmap e695: 8 total, 8 up, 8 in 2024-09-21T12:08:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:08:29.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:08:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:08:30.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:29 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:08:30.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:08:30.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:29 smithi107 ceph-mon[41097]: osdmap e696: 8 total, 8 up, 8 in 2024-09-21T12:08:30.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:29 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:08:30.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:08:30.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:29 smithi107 ceph-mon[30293]: osdmap e696: 8 total, 8 up, 8 in 2024-09-21T12:08:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:29 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:08:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:08:30.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:29 smithi193 ceph-mon[33333]: osdmap e696: 8 total, 8 up, 8 in 2024-09-21T12:08:31.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:31.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T12:08:31.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: osdmap e697: 8 total, 8 up, 8 in 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: osdmap e697: 8 total, 8 up, 8 in 2024-09-21T12:08:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: osdmap e697: 8 total, 8 up, 8 in 2024-09-21T12:08:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_grade_decay_rate","val": "20"} : dispatch 2024-09-21T12:08:32.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:08:32.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[30293]: osdmap e698: 8 total, 8 up, 8 in 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[41097]: osdmap e698: 8 total, 8 up, 8 in 2024-09-21T12:08:32.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-21T12:08:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:31 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:08:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:31 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:08:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:31 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:08:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-21T12:08:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:31 smithi193 ceph-mon[33333]: osdmap e698: 8 total, 8 up, 8 in 2024-09-21T12:08:32.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_search_last_n","val": "1"} : dispatch 2024-09-21T12:08:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:08:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:08:33.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[30293]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T12:08:33.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-21T12:08:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[30293]: osdmap e699: 8 total, 8 up, 8 in 2024-09-21T12:08:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[41097]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T12:08:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-21T12:08:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[41097]: osdmap e699: 8 total, 8 up, 8 in 2024-09-21T12:08:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:32 smithi193 ceph-mon[33333]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-21T12:08:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-21T12:08:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:32 smithi193 ceph-mon[33333]: osdmap e699: 8 total, 8 up, 8 in 2024-09-21T12:08:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[30293]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:08:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[30293]: osdmap e700: 8 total, 8 up, 8 in 2024-09-21T12:08:35.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[41097]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:08:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[41097]: osdmap e700: 8 total, 8 up, 8 in 2024-09-21T12:08:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:34 smithi193 ceph-mon[33333]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:08:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:34 smithi193 ceph-mon[33333]: osdmap e700: 8 total, 8 up, 8 in 2024-09-21T12:08:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:36.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"}]': finished 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[30293]: osdmap e701: 8 total, 8 up, 8 in 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"}]': finished 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[41097]: osdmap e701: 8 total, 8 up, 8 in 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:36.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"}]': finished 2024-09-21T12:08:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:35 smithi193 ceph-mon[33333]: osdmap e701: 8 total, 8 up, 8 in 2024-09-21T12:08:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:36.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-142"} : dispatch 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[30293]: pgmap v1074: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[30293]: osdmap e702: 8 total, 8 up, 8 in 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[30293]: osdmap e703: 8 total, 8 up, 8 in 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[41097]: pgmap v1074: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[41097]: osdmap e702: 8 total, 8 up, 8 in 2024-09-21T12:08:37.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[41097]: osdmap e703: 8 total, 8 up, 8 in 2024-09-21T12:08:37.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:36 smithi193 ceph-mon[33333]: pgmap v1074: 204 pgs: 204 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:08:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:36 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:36 smithi193 ceph-mon[33333]: osdmap e702: 8 total, 8 up, 8 in 2024-09-21T12:08:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:36 smithi193 ceph-mon[33333]: osdmap e703: 8 total, 8 up, 8 in 2024-09-21T12:08:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-144","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:39.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[30293]: pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[30293]: osdmap e704: 8 total, 8 up, 8 in 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[41097]: pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[41097]: osdmap e704: 8 total, 8 up, 8 in 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:38 smithi193 ceph-mon[33333]: pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 714 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:38 smithi193 ceph-mon[33333]: osdmap e704: 8 total, 8 up, 8 in 2024-09-21T12:08:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:38 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:39.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:08:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:08:40.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:40.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:39 smithi107 ceph-mon[30293]: osdmap e705: 8 total, 8 up, 8 in 2024-09-21T12:08:40.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:39 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:40.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:40.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:39 smithi107 ceph-mon[41097]: osdmap e705: 8 total, 8 up, 8 in 2024-09-21T12:08:40.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:39 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:40.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:40.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:39 smithi193 ceph-mon[33333]: osdmap e705: 8 total, 8 up, 8 in 2024-09-21T12:08:40.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:39 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:41.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:41.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[41097]: pgmap v1080: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-144"}]': finished 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[41097]: osdmap e706: 8 total, 8 up, 8 in 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-144", "mode": "readproxy"} : dispatch 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[30293]: pgmap v1080: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-144"}]': finished 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[30293]: osdmap e706: 8 total, 8 up, 8 in 2024-09-21T12:08:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:40 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-144", "mode": "readproxy"} : dispatch 2024-09-21T12:08:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:40 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:40 smithi193 ceph-mon[33333]: pgmap v1080: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:08:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-144"}]': finished 2024-09-21T12:08:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:40 smithi193 ceph-mon[33333]: osdmap e706: 8 total, 8 up, 8 in 2024-09-21T12:08:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:40 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-144", "mode": "readproxy"} : dispatch 2024-09-21T12:08:42.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:41 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:41 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-144", "mode": "readproxy"}]': finished 2024-09-21T12:08:42.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:41 smithi107 ceph-mon[30293]: osdmap e707: 8 total, 8 up, 8 in 2024-09-21T12:08:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:41 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:41 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-144", "mode": "readproxy"}]': finished 2024-09-21T12:08:42.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:41 smithi107 ceph-mon[41097]: osdmap e707: 8 total, 8 up, 8 in 2024-09-21T12:08:42.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:41 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:42.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:41 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-144", "mode": "readproxy"}]': finished 2024-09-21T12:08:42.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:41 smithi193 ceph-mon[33333]: osdmap e707: 8 total, 8 up, 8 in 2024-09-21T12:08:42.697 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:08:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:08:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:08:43.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:42 smithi107 ceph-mon[30293]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:08:43.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:42 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:42 smithi107 ceph-mon[41097]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:08:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:42 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:43.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:42 smithi193 ceph-mon[33333]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:08:43.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:42 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:45.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:44 smithi107 ceph-mon[30293]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 174 B/s wr, 2 op/s 2024-09-21T12:08:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:44 smithi107 ceph-mon[41097]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 174 B/s wr, 2 op/s 2024-09-21T12:08:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:44 smithi193 ceph-mon[33333]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 174 B/s wr, 2 op/s 2024-09-21T12:08:47.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:46 smithi107 ceph-mon[30293]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 894 B/s rd, 0 op/s 2024-09-21T12:08:47.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:46 smithi107 ceph-mon[41097]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 894 B/s rd, 0 op/s 2024-09-21T12:08:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:46 smithi193 ceph-mon[33333]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 894 B/s rd, 0 op/s 2024-09-21T12:08:49.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:48 smithi107 ceph-mon[30293]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:08:49.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:48 smithi107 ceph-mon[41097]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:08:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:48 smithi193 ceph-mon[33333]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:08:49.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:08:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:08:51.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:50 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:51.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:50 smithi107 ceph-mon[30293]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:08:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:50 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:51.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:50 smithi107 ceph-mon[41097]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:08:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:50 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:08:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:50 smithi193 ceph-mon[33333]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:08:52.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:52.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:52.728 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:08:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:08:53.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[30293]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[30293]: osdmap e708: 8 total, 8 up, 8 in 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[41097]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[41097]: osdmap e708: 8 total, 8 up, 8 in 2024-09-21T12:08:53.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:52 smithi193 ceph-mon[33333]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-21T12:08:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:08:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:52 smithi193 ceph-mon[33333]: osdmap e708: 8 total, 8 up, 8 in 2024-09-21T12:08:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:54.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:08:54.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"}]': finished 2024-09-21T12:08:54.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[30293]: osdmap e709: 8 total, 8 up, 8 in 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"}]': finished 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[41097]: osdmap e709: 8 total, 8 up, 8 in 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:54.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:53 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:08:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"}]': finished 2024-09-21T12:08:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:53 smithi193 ceph-mon[33333]: osdmap e709: 8 total, 8 up, 8 in 2024-09-21T12:08:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:08:54.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-144"} : dispatch 2024-09-21T12:08:54.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:08:55.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:54 smithi107 ceph-mon[30293]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:54 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:54 smithi107 ceph-mon[30293]: osdmap e710: 8 total, 8 up, 8 in 2024-09-21T12:08:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:54 smithi107 ceph-mon[41097]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:54 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:55.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:54 smithi107 ceph-mon[41097]: osdmap e710: 8 total, 8 up, 8 in 2024-09-21T12:08:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:54 smithi193 ceph-mon[33333]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:54 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:08:55.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:54 smithi193 ceph-mon[33333]: osdmap e710: 8 total, 8 up, 8 in 2024-09-21T12:08:56.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:55 smithi107 ceph-mon[30293]: osdmap e711: 8 total, 8 up, 8 in 2024-09-21T12:08:56.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:56.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:55 smithi107 ceph-mon[41097]: osdmap e711: 8 total, 8 up, 8 in 2024-09-21T12:08:56.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:55 smithi193 ceph-mon[33333]: osdmap e711: 8 total, 8 up, 8 in 2024-09-21T12:08:56.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-146","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:08:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:56 smithi193 ceph-mon[33333]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:56 smithi193 ceph-mon[33333]: osdmap e712: 8 total, 8 up, 8 in 2024-09-21T12:08:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:57.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[41097]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[41097]: osdmap e712: 8 total, 8 up, 8 in 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[30293]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[30293]: osdmap e712: 8 total, 8 up, 8 in 2024-09-21T12:08:57.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:08:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:57 smithi193 ceph-mon[33333]: osdmap e713: 8 total, 8 up, 8 in 2024-09-21T12:08:58.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:08:58.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-146"}]': finished 2024-09-21T12:08:58.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:57 smithi193 ceph-mon[33333]: osdmap e714: 8 total, 8 up, 8 in 2024-09-21T12:08:58.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-146", "mode": "writeback"} : dispatch 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[30293]: osdmap e713: 8 total, 8 up, 8 in 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-146"}]': finished 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[30293]: osdmap e714: 8 total, 8 up, 8 in 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-146", "mode": "writeback"} : dispatch 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[41097]: osdmap e713: 8 total, 8 up, 8 in 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:08:58.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-77755-111", "overlaypool": "test-rados-api-smithi107-77755-146"}]': finished 2024-09-21T12:08:58.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[41097]: osdmap e714: 8 total, 8 up, 8 in 2024-09-21T12:08:58.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-146", "mode": "writeback"} : dispatch 2024-09-21T12:08:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:58 smithi193 ceph-mon[33333]: pgmap v1097: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:58 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-146", "mode": "writeback"}]': finished 2024-09-21T12:08:59.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:58 smithi193 ceph-mon[33333]: osdmap e715: 8 total, 8 up, 8 in 2024-09-21T12:08:59.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[30293]: pgmap v1097: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-146", "mode": "writeback"}]': finished 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[30293]: osdmap e715: 8 total, 8 up, 8 in 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[41097]: pgmap v1097: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-77755-146", "mode": "writeback"}]': finished 2024-09-21T12:08:59.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:58 smithi107 ceph-mon[41097]: osdmap e715: 8 total, 8 up, 8 in 2024-09-21T12:08:59.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:08:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:09:00.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:08:59 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:09:00.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:08:59 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:09:00.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:08:59 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_count","val": "2"} : dispatch 2024-09-21T12:09:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:00 smithi193 ceph-mon[33333]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:09:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:09:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:00 smithi193 ceph-mon[33333]: osdmap e716: 8 total, 8 up, 8 in 2024-09-21T12:09:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:00 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:09:01.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[30293]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[30293]: osdmap e716: 8 total, 8 up, 8 in 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[41097]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_count","val": "2"}]': finished 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[41097]: osdmap e716: 8 total, 8 up, 8 in 2024-09-21T12:09:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:00 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_period","val": "600"} : dispatch 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: osdmap e717: 8 total, 8 up, 8 in 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: osdmap e718: 8 total, 8 up, 8 in 2024-09-21T12:09:02.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:01 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:09:02.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:09:02.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: osdmap e717: 8 total, 8 up, 8 in 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: osdmap e718: 8 total, 8 up, 8 in 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_period","val": "600"}]': finished 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: osdmap e717: 8 total, 8 up, 8 in 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_type","val": "bloom"} : dispatch 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: osdmap e718: 8 total, 8 up, 8 in 2024-09-21T12:09:02.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:01 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "min_read_recency_for_promote","val": "1"} : dispatch 2024-09-21T12:09:02.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:09:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:09:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:02 smithi193 ceph-mon[33333]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:09:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:02 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:09:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:02 smithi193 ceph-mon[33333]: osdmap e719: 8 total, 8 up, 8 in 2024-09-21T12:09:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:02 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[30293]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[30293]: osdmap e719: 8 total, 8 up, 8 in 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[41097]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[41097]: osdmap e719: 8 total, 8 up, 8 in 2024-09-21T12:09:03.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:02 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "target_max_objects","val": "1"} : dispatch 2024-09-21T12:09:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:04 smithi193 ceph-mon[33333]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:09:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:04 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "target_max_objects","val": "1"}]': finished 2024-09-21T12:09:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:04 smithi193 ceph-mon[33333]: osdmap e720: 8 total, 8 up, 8 in 2024-09-21T12:09:05.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:04 smithi107 ceph-mon[30293]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:09:05.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:04 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "target_max_objects","val": "1"}]': finished 2024-09-21T12:09:05.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:04 smithi107 ceph-mon[30293]: osdmap e720: 8 total, 8 up, 8 in 2024-09-21T12:09:05.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:04 smithi107 ceph-mon[41097]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:09:05.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:04 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-146","var": "target_max_objects","val": "1"}]': finished 2024-09-21T12:09:05.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:04 smithi107 ceph-mon[41097]: osdmap e720: 8 total, 8 up, 8 in 2024-09-21T12:09:06.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:05 smithi107 ceph-mon[30293]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-21T12:09:06.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:05 smithi107 ceph-mon[41097]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-21T12:09:06.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:05 smithi193 ceph-mon[33333]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-21T12:09:07.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:06 smithi107 ceph-mon[30293]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:09:07.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:06 smithi107 ceph-mon[41097]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:09:07.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:06 smithi193 ceph-mon[33333]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-21T12:09:09.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:08 smithi107 ceph-mon[41097]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:08 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:08 smithi107 ceph-mon[30293]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:08 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:09.279 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:08 smithi193 ceph-mon[33333]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:09.279 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:08 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:09.280 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:09.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:09:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:09:11.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:11.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:10 smithi107 ceph-mon[41097]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:10 smithi107 ceph-mon[30293]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:10 smithi193 ceph-mon[33333]: pgmap v1110: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:09:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:09:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:09:13.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:12 smithi107 ceph-mon[41097]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:12 smithi107 ceph-mon[30293]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:12 smithi193 ceph-mon[33333]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-21T12:09:14.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:14.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:14.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[41097]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[41097]: osdmap e721: 8 total, 8 up, 8 in 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[30293]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[30293]: osdmap e721: 8 total, 8 up, 8 in 2024-09-21T12:09:15.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:09:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:14 smithi193 ceph-mon[33333]: pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:09:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:14 smithi193 ceph-mon[33333]: osdmap e721: 8 total, 8 up, 8 in 2024-09-21T12:09:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"}]': finished 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[30293]: osdmap e722: 8 total, 8 up, 8 in 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"}]': finished 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[41097]: osdmap e722: 8 total, 8 up, 8 in 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:16.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:09:16.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"}]': finished 2024-09-21T12:09:16.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:15 smithi193 ceph-mon[33333]: osdmap e722: 8 total, 8 up, 8 in 2024-09-21T12:09:16.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:16.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-146"} : dispatch 2024-09-21T12:09:17.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[30293]: pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[30293]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[30293]: osdmap e723: 8 total, 8 up, 8 in 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[41097]: pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[41097]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-21T12:09:17.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:16 smithi107 ceph-mon[41097]: osdmap e723: 8 total, 8 up, 8 in 2024-09-21T12:09:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:16 smithi193 ceph-mon[33333]: pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-21T12:09:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:16 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:16 smithi193 ceph-mon[33333]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-21T12:09:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:16 smithi193 ceph-mon[33333]: osdmap e723: 8 total, 8 up, 8 in 2024-09-21T12:09:18.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:17 smithi107 ceph-mon[30293]: osdmap e724: 8 total, 8 up, 8 in 2024-09-21T12:09:18.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:17 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:17 smithi107 ceph-mon[41097]: osdmap e724: 8 total, 8 up, 8 in 2024-09-21T12:09:18.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:17 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:17 smithi193 ceph-mon[33333]: osdmap e724: 8 total, 8 up, 8 in 2024-09-21T12:09:18.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:17 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-148","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:19.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[30293]: pgmap v1118: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:19.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:19.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[30293]: osdmap e725: 8 total, 8 up, 8 in 2024-09-21T12:09:19.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:09:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[41097]: pgmap v1118: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[41097]: osdmap e725: 8 total, 8 up, 8 in 2024-09-21T12:09:19.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:18 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:09:19.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:18 smithi193 ceph-mon[33333]: pgmap v1118: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:19.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:19.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:18 smithi193 ceph-mon[33333]: osdmap e725: 8 total, 8 up, 8 in 2024-09-21T12:09:19.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:18 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148", "force_nonempty": "--force-nonempty" } : dispatch 2024-09-21T12:09:19.550 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:09:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:09:20.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:09:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:19 smithi107 ceph-mon[30293]: osdmap e726: 8 total, 8 up, 8 in 2024-09-21T12:09:20.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:19 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"} : dispatch 2024-09-21T12:09:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:09:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:19 smithi107 ceph-mon[41097]: osdmap e726: 8 total, 8 up, 8 in 2024-09-21T12:09:20.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:19 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"} : dispatch 2024-09-21T12:09:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-21T12:09:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:19 smithi193 ceph-mon[33333]: osdmap e726: 8 total, 8 up, 8 in 2024-09-21T12:09:20.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:19 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"} : dispatch 2024-09-21T12:09:21.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[41097]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"}]': finished 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[41097]: osdmap e727: 8 total, 8 up, 8 in 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"} : dispatch 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[30293]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"}]': finished 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[30293]: osdmap e727: 8 total, 8 up, 8 in 2024-09-21T12:09:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:21.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:20 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"} : dispatch 2024-09-21T12:09:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:20 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:20 smithi193 ceph-mon[33333]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-21T12:09:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"}]': finished 2024-09-21T12:09:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:20 smithi193 ceph-mon[33333]: osdmap e727: 8 total, 8 up, 8 in 2024-09-21T12:09:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:20 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-148"} : dispatch 2024-09-21T12:09:22.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:21 smithi107 ceph-mon[41097]: osdmap e728: 8 total, 8 up, 8 in 2024-09-21T12:09:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:21 smithi107 ceph-mon[41097]: osdmap e729: 8 total, 8 up, 8 in 2024-09-21T12:09:22.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:21 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:21 smithi107 ceph-mon[30293]: osdmap e728: 8 total, 8 up, 8 in 2024-09-21T12:09:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:21 smithi107 ceph-mon[30293]: osdmap e729: 8 total, 8 up, 8 in 2024-09-21T12:09:22.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:21 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:22.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:21 smithi193 ceph-mon[33333]: osdmap e728: 8 total, 8 up, 8 in 2024-09-21T12:09:22.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:21 smithi193 ceph-mon[33333]: osdmap e729: 8 total, 8 up, 8 in 2024-09-21T12:09:22.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:21 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-150","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:22.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:09:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[30293]: pgmap v1124: 172 pgs: 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[30293]: osdmap e730: 8 total, 8 up, 8 in 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[41097]: pgmap v1124: 172 pgs: 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:23.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:22 smithi107 ceph-mon[41097]: osdmap e730: 8 total, 8 up, 8 in 2024-09-21T12:09:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:22 smithi193 ceph-mon[33333]: pgmap v1124: 172 pgs: 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-21T12:09:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:22 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:22 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:22 smithi193 ceph-mon[33333]: osdmap e730: 8 total, 8 up, 8 in 2024-09-21T12:09:24.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-150"} : dispatch 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[30293]: osdmap e731: 8 total, 8 up, 8 in 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-150"} : dispatch 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:24.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:23 smithi107 ceph-mon[41097]: osdmap e731: 8 total, 8 up, 8 in 2024-09-21T12:09:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:23 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-150"} : dispatch 2024-09-21T12:09:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:23 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:23 smithi193 ceph-mon[33333]: osdmap e731: 8 total, 8 up, 8 in 2024-09-21T12:09:25.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:24 smithi107 ceph-mon[30293]: pgmap v1127: 204 pgs: 3 unknown, 201 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-21T12:09:25.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:24 smithi107 ceph-mon[30293]: osdmap e732: 8 total, 8 up, 8 in 2024-09-21T12:09:25.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:24 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:24 smithi107 ceph-mon[41097]: pgmap v1127: 204 pgs: 3 unknown, 201 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-21T12:09:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:24 smithi107 ceph-mon[41097]: osdmap e732: 8 total, 8 up, 8 in 2024-09-21T12:09:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:24 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:25.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:24 smithi193 ceph-mon[33333]: pgmap v1127: 204 pgs: 3 unknown, 201 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-21T12:09:25.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:24 smithi193 ceph-mon[33333]: osdmap e732: 8 total, 8 up, 8 in 2024-09-21T12:09:25.302 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:24 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-152","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:27.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[41097]: pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[41097]: osdmap e733: 8 total, 8 up, 8 in 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-152"} : dispatch 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[30293]: pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[30293]: osdmap e733: 8 total, 8 up, 8 in 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:27.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:26 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-152"} : dispatch 2024-09-21T12:09:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:26 smithi193 ceph-mon[33333]: pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-21T12:09:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:26 smithi193 ceph-mon[33333]: osdmap e733: 8 total, 8 up, 8 in 2024-09-21T12:09:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:26 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-152"} : dispatch 2024-09-21T12:09:28.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[30293]: osdmap e734: 8 total, 8 up, 8 in 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[30293]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[30293]: osdmap e735: 8 total, 8 up, 8 in 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[41097]: osdmap e734: 8 total, 8 up, 8 in 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[41097]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[41097]: osdmap e735: 8 total, 8 up, 8 in 2024-09-21T12:09:28.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:27 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:27 smithi193 ceph-mon[33333]: osdmap e734: 8 total, 8 up, 8 in 2024-09-21T12:09:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:27 smithi193 ceph-mon[33333]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:27 smithi193 ceph-mon[33333]: osdmap e735: 8 total, 8 up, 8 in 2024-09-21T12:09:28.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:27 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-154","app": "rados","yes_i_really_mean_it": true} : dispatch 2024-09-21T12:09:29.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[41097]: pgmap v1133: 172 pgs: 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:29.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[41097]: osdmap e736: 8 total, 8 up, 8 in 2024-09-21T12:09:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-111","var": "dedup_tier","val": "test-rados-api-smithi107-77755-154"} : dispatch 2024-09-21T12:09:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[30293]: pgmap v1133: 172 pgs: 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[30293]: osdmap e736: 8 total, 8 up, 8 in 2024-09-21T12:09:29.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:28 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-111","var": "dedup_tier","val": "test-rados-api-smithi107-77755-154"} : dispatch 2024-09-21T12:09:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:28 smithi193 ceph-mon[33333]: pgmap v1133: 172 pgs: 172 active+clean; 4.4 MiB data, 797 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-77755-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-21T12:09:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:28 smithi193 ceph-mon[33333]: osdmap e736: 8 total, 8 up, 8 in 2024-09-21T12:09:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:28 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd pool set","pool":"test-rados-api-smithi107-77755-111","var": "dedup_tier","val": "test-rados-api-smithi107-77755-154"} : dispatch 2024-09-21T12:09:29.801 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:09:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:09:30.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:30.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-154"} : dispatch 2024-09-21T12:09:30.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:29 smithi107 ceph-mon[30293]: osdmap e737: 8 total, 8 up, 8 in 2024-09-21T12:09:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-154"} : dispatch 2024-09-21T12:09:30.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:29 smithi107 ceph-mon[41097]: osdmap e737: 8 total, 8 up, 8 in 2024-09-21T12:09:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-77755-111", "tierpool": "test-rados-api-smithi107-77755-154"} : dispatch 2024-09-21T12:09:30.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:29 smithi193 ceph-mon[33333]: osdmap e737: 8 total, 8 up, 8 in 2024-09-21T12:09:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[30293]: pgmap v1136: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T12:09:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[30293]: osdmap e738: 8 total, 8 up, 8 in 2024-09-21T12:09:31.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[41097]: pgmap v1136: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T12:09:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[41097]: osdmap e738: 8 total, 8 up, 8 in 2024-09-21T12:09:31.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:30 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:30 smithi193 ceph-mon[33333]: pgmap v1136: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 819 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T12:09:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:30 smithi193 ceph-mon[33333]: osdmap e738: 8 total, 8 up, 8 in 2024-09-21T12:09:31.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:32.201 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: osdmap e739: 8 total, 8 up, 8 in 2024-09-21T12:09:32.202 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:32.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: osdmap e739: 8 total, 8 up, 8 in 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:09:32.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: osdmap e739: 8 total, 8 up, 8 in 2024-09-21T12:09:32.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd={"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"} : dispatch 2024-09-21T12:09:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:09:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:09:32.940 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7809 ms) 2024-09-21T12:09:32.940 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12160 ms) 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8038 ms) 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13073 ms) 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7747 ms) 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8064 ms) 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-21T12:09:32.941 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9042 ms) 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first is 1726920488 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,1726920493,1726920494,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,1726920493,1726920494,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,1726920493,1726920494,0 2024-09-21T12:09:32.942 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,1726920493,1726920494,1726920496,1726920497,0 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,1726920493,1726920494,1726920496,1726920497,0 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920488,1726920490,1726920491,1726920493,1726920494,1726920496,1726920497,0 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1726920491,1726920493,1726920494,1726920496,1726920497,1726920499,1726920500,0 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first now 1726920491, trimmed 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21070 ms) 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: foo0 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14039 ms) 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18076 ms) 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22137 ms) 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-21T12:09:32.943 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5028 ms) 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3009 ms) 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3005 ms) 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3008 ms) 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (229944 ms total) 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (843177 ms total) 2024-09-21T12:09:32.944 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-21T12:09:32.948 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77414 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77414 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77635 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77635 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78162 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78162 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77888 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77888 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77752 2024-09-21T12:09:32.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77752 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77490 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77490 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77830 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77830 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78316 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78316 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78389 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78389 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77510 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77510 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78192 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78192 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77386 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77386 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77451 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77451 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78119 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78119 2024-09-21T12:09:32.951 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77377 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77377 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77399 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77399 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78618 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78618 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77674 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77674 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78695 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78695 2024-09-21T12:09:32.952 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77799 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77799 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78250 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78250 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78742 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78742 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77707 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77707 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78127 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78127 2024-09-21T12:09:32.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78535 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78535 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78232 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78232 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78646 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78646 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78078 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78078 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77567 2024-09-21T12:09:32.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77567 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78725 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78725 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77915 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77915 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77370 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77370 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77430 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77430 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.955 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77851 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77851 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77470 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77470 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=77956 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 77956 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78025 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78025 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78426 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78426 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-21T12:09:32.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=78763 2024-09-21T12:09:32.957 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 78763 2024-09-21T12:09:32.957 INFO:tasks.workunit.client.0.smithi107.stderr:+ exit 0 2024-09-21T12:09:32.957 INFO:tasks.workunit.client.0.smithi107.stderr:+ cleanup 2024-09-21T12:09:32.957 INFO:tasks.workunit.client.0.smithi107.stderr:+ pkill -P 77364 2024-09-21T12:09:32.964 INFO:tasks.workunit.client.0.smithi107.stderr:+ true 2024-09-21T12:09:32.965 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T12:09:32.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T12:09:32.995 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-21T12:09:32.997 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-21T12:09:32.997 DEBUG:teuthology.orchestra.run.smithi107:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-21T12:09:33.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ uuidgen 2024-09-21T12:09:33.066 INFO:tasks.workunit.client.0.smithi107.stderr:+ p=abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:09:33.066 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool create abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 12 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 -- 172.21.15.107:0/2121547875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 msgr2=0x7fcb50101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2121547875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50101fd0 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009860 tx=0x7fcb4c02a340 comp rx=0 tx=0).stop 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 -- 172.21.15.107:0/2121547875 shutdown_connections 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2121547875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb50107f70 0x7fcb5010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2121547875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb501075f0 0x7fcb50107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2121547875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50101fd0 unknown :-1 s=CLOSED pgs=3000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:33.151 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 -- 172.21.15.107:0/2121547875 >> 172.21.15.107:0/2121547875 conn(0x7fcb500fd650 msgr2=0x7fcb500ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 -- 172.21.15.107:0/2121547875 shutdown_connections 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.147+0000 7fcb582f6640 1 -- 172.21.15.107:0/2121547875 wait complete. 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 Processor -- start 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 -- start start 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb501075f0 0x7fcb50198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb50107f70 0x7fcb5019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111100 con 0x7fcb50101bf0 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50110f80 con 0x7fcb50107f70 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb582f6640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fcb50111280 con 0x7fcb501075f0 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:33.152 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb55891640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb501075f0 0x7fcb50198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47136/0 (socket says 172.21.15.107:47136) 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 -- 172.21.15.107:0/2904097606 learned_addr learned my addr 172.21.15.107:0/2904097606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 -- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb501075f0 msgr2=0x7fcb50198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb501075f0 0x7fcb50198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 -- 172.21.15.107:0/2904097606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb50107f70 msgr2=0x7fcb5019e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb50107f70 0x7fcb5019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 -- 172.21.15.107:0/2904097606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5019e8e0 con 0x7fcb50101bf0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb55891640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb501075f0 0x7fcb50198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.148+0000 7fcb56092640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50198840 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c035040 tx=0x7fcb4c033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.149+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c03f020 con 0x7fcb50101bf0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.149+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fcb4c0306b0 con 0x7fcb50101bf0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.149+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5019eb70 con 0x7fcb50101bf0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.149+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c037040 con 0x7fcb50101bf0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.149+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501a5310 con 0x7fcb50101bf0 2024-09-21T12:09:33.153 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.150+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb50102e90 con 0x7fcb50101bf0 2024-09-21T12:09:33.154 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.150+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7fcb4c042020 con 0x7fcb50101bf0 2024-09-21T12:09:33.154 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.151+0000 7fcb477fe640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fcb3c07b150 0x7fcb3c07d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:33.154 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.151+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(740..740 src has 1..740) ==== 8675+0+0 (secure 0 0 0) 0x7fcb4c0beed0 con 0x7fcb50101bf0 2024-09-21T12:09:33.154 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.151+0000 7fcb55891640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fcb3c07b150 0x7fcb3c07d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:33.154 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.151+0000 7fcb55891640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fcb3c07b150 0x7fcb3c07d610 secure :-1 s=READY pgs=3927 cs=0 l=1 rev1=1 crypto rx=0x7fcb5006a580 tx=0x7fcb40009110 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:09:33.156 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.155+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb4c049020 con 0x7fcb50101bf0 2024-09-21T12:09:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[41097]: pgmap v1139: 164 pgs: 164 active+clean; 583 KiB data, 819 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[41097]: osdmap e740: 8 total, 8 up, 8 in 2024-09-21T12:09:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[30293]: pgmap v1139: 164 pgs: 164 active+clean; 583 KiB data, 819 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:32 smithi107 ceph-mon[30293]: osdmap e740: 8 total, 8 up, 8 in 2024-09-21T12:09:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:32 smithi193 ceph-mon[33333]: pgmap v1139: 164 pgs: 164 active+clean; 583 KiB data, 819 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:32 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:32 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3224587184' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-77755-111"}]': finished 2024-09-21T12:09:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:32 smithi193 ceph-mon[33333]: osdmap e740: 8 total, 8 up, 8 in 2024-09-21T12:09:33.337 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.336+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} v 0) -- 0x7fcb50101fd0 con 0x7fcb50101bf0 2024-09-21T12:09:33.955 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:33.954+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12}]=0 pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' created v741) ==== 176+0+0 (secure 0 0 0) 0x7fcb4c089ba0 con 0x7fcb50101bf0 2024-09-21T12:09:34.067 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.066+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} v 0) -- 0x7fcb501a5540 con 0x7fcb50101bf0 2024-09-21T12:09:34.068 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.066+0000 7fcb477fe640 1 -- 172.21.15.107:0/2904097606 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12}]=0 pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' already exists v741) ==== 183+0+0 (secure 0 0 0) 0x7fcb4c08ea50 con 0x7fcb50101bf0 2024-09-21T12:09:34.068 INFO:tasks.workunit.client.0.smithi107.stderr:pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' already exists 2024-09-21T12:09:34.070 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fcb3c07b150 msgr2=0x7fcb3c07d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:34.070 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fcb3c07b150 0x7fcb3c07d610 secure :-1 s=READY pgs=3927 cs=0 l=1 rev1=1 crypto rx=0x7fcb5006a580 tx=0x7fcb40009110 comp rx=0 tx=0).stop 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 msgr2=0x7fcb50198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50198840 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c035040 tx=0x7fcb4c033d00 comp rx=0 tx=0).stop 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 shutdown_connections 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fcb3c07b150 0x7fcb3c07d610 unknown :-1 s=CLOSED pgs=3927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb50107f70 0x7fcb5019e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb501075f0 0x7fcb50198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 --2- 172.21.15.107:0/2904097606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb50101bf0 0x7fcb50198840 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.068+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 >> 172.21.15.107:0/2904097606 conn(0x7fcb500fd650 msgr2=0x7fcb50109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.069+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 shutdown_connections 2024-09-21T12:09:34.071 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.069+0000 7fcb582f6640 1 -- 172.21.15.107:0/2904097606 wait complete. 2024-09-21T12:09:34.079 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_objects 10 2024-09-21T12:09:34.165 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 -- 172.21.15.107:0/1668437508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc1043b0 msgr2=0x7f92fc104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:34.165 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 --2- 172.21.15.107:0/1668437508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc1043b0 0x7f92fc104810 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009900 tx=0x7f92e402a410 comp rx=0 tx=0).stop 2024-09-21T12:09:34.165 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 -- 172.21.15.107:0/1668437508 shutdown_connections 2024-09-21T12:09:34.165 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 --2- 172.21.15.107:0/1668437508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc104d50 0x7f92fc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.165 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 --2- 172.21.15.107:0/1668437508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc1043b0 0x7f92fc104810 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 --2- 172.21.15.107:0/1668437508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92fc101f90 0x7f92fc102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 -- 172.21.15.107:0/1668437508 >> 172.21.15.107:0/1668437508 conn(0x7f92fc0776a0 msgr2=0x7f92fc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 -- 172.21.15.107:0/1668437508 shutdown_connections 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 -- 172.21.15.107:0/1668437508 wait complete. 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 Processor -- start 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.161+0000 7f930133d640 1 -- start start 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 0x7f92fc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc1043b0 0x7f92fc199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92fc104d50 0x7f92fc1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:34.166 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc1110e0 con 0x7f92fc101f90 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc110f60 con 0x7f92fc104d50 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc111260 con 0x7f92fc1043b0 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92faffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 0x7f92fc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc1043b0 0x7f92fc199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92faffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 0x7f92fc1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42540/0 (socket says 172.21.15.107:42540) 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92faffd640 1 -- 172.21.15.107:0/3163807476 learned_addr learned my addr 172.21.15.107:0/3163807476 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92fb7fe640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92fc104d50 0x7f92fc1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 -- 172.21.15.107:0/3163807476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92fc104d50 msgr2=0x7f92fc1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92fc104d50 0x7f92fc1a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 -- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 msgr2=0x7f92fc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 0x7f92fc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 -- 172.21.15.107:0/3163807476 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc1a4b60 con 0x7f92fc1043b0 2024-09-21T12:09:34.167 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92faffd640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 0x7f92fc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f3fff640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc1043b0 0x7f92fc199ae0 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7f92e402a930 tx=0x7f92e4032af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e403f030 con 0x7f92fc1043b0 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f92e4003a40 con 0x7f92fc1043b0 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc1a4d30 con 0x7f92fc1043b0 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4035070 con 0x7f92fc1043b0 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.162+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc1a5190 con 0x7f92fc1043b0 2024-09-21T12:09:34.168 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.163+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc102370 con 0x7f92fc1043b0 2024-09-21T12:09:34.169 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.164+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f92e4042030 con 0x7f92fc1043b0 2024-09-21T12:09:34.169 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.164+0000 7f92f8ff9640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f92e807b150 0x7f92e807d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:34.169 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.164+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(741..741 src has 1..741) ==== 9050+0+0 (secure 0 0 0) 0x7f92e40bf1f0 con 0x7f92fc1043b0 2024-09-21T12:09:34.169 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.164+0000 7f92faffd640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f92e807b150 0x7f92e807d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:34.169 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.165+0000 7f92faffd640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f92e807b150 0x7f92e807d610 secure :-1 s=READY pgs=3928 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e10 tx=0x7f92ec0091c0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:09:34.170 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.169+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92e4089c50 con 0x7f92fc1043b0 2024-09-21T12:09:34.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:33 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} : dispatch 2024-09-21T12:09:34.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:33 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} : dispatch 2024-09-21T12:09:34.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:33 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} : dispatch 2024-09-21T12:09:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.348+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} v 0) -- 0x7f92fc1a5550 con 0x7f92fc1043b0 2024-09-21T12:09:34.963 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:34.962+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v742) ==== 223+0+0 (secure 0 0 0) 0x7f92e408eb00 con 0x7f92fc1043b0 2024-09-21T12:09:35.076 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.075+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} v 0) -- 0x7f92fc1a57a0 con 0x7f92fc1043b0 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[30293]: pgmap v1142: 164 pgs: 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12}]': finished 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[30293]: osdmap e741: 8 total, 8 up, 8 in 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} : dispatch 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3163807476' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[41097]: pgmap v1142: 164 pgs: 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12}]': finished 2024-09-21T12:09:35.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[41097]: osdmap e741: 8 total, 8 up, 8 in 2024-09-21T12:09:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} : dispatch 2024-09-21T12:09:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3163807476' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:35.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:34 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:34 smithi193 ceph-mon[33333]: pgmap v1142: 164 pgs: 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12}]': finished 2024-09-21T12:09:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:34 smithi193 ceph-mon[33333]: osdmap e741: 8 total, 8 up, 8 in 2024-09-21T12:09:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2904097606' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pg_num": 12} : dispatch 2024-09-21T12:09:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:34 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3163807476' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:34 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:35.980 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.978+0000 7f92f8ff9640 1 -- 172.21.15.107:0/3163807476 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v743) ==== 223+0+0 (secure 0 0 0) 0x7f92e40bcaa0 con 0x7f92fc1043b0 2024-09-21T12:09:35.980 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 10 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f92e807b150 msgr2=0x7f92e807d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f92e807b150 0x7f92e807d610 secure :-1 s=READY pgs=3928 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e10 tx=0x7f92ec0091c0 comp rx=0 tx=0).stop 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc1043b0 msgr2=0x7f92fc199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc1043b0 0x7f92fc199ae0 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7f92e402a930 tx=0x7f92e4032af0 comp rx=0 tx=0).stop 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 shutdown_connections 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f92e807b150 0x7f92e807d610 unknown :-1 s=CLOSED pgs=3928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92fc104d50 0x7f92fc1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92fc1043b0 0x7f92fc199ae0 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:35.983 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 --2- 172.21.15.107:0/3163807476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92fc101f90 0x7f92fc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:35.984 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 >> 172.21.15.107:0/3163807476 conn(0x7f92fc0776a0 msgr2=0x7f92fc1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:09:35.984 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 shutdown_connections 2024-09-21T12:09:35.984 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:35.981+0000 7f930133d640 1 -- 172.21.15.107:0/3163807476 wait complete. 2024-09-21T12:09:35.994 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool application enable abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 rados 2024-09-21T12:09:36.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 -- 172.21.15.107:0/1408813489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c108700 msgr2=0x7f706c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:36.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 --2- 172.21.15.107:0/1408813489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c108700 0x7f706c108b40 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f705c009860 tx=0x7f705c02a340 comp rx=0 tx=0).stop 2024-09-21T12:09:36.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 -- 172.21.15.107:0/1408813489 shutdown_connections 2024-09-21T12:09:36.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 --2- 172.21.15.107:0/1408813489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c109080 0x7f706c10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 --2- 172.21.15.107:0/1408813489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c108700 0x7f706c108b40 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 --2- 172.21.15.107:0/1408813489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c102d00 0x7f706c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 -- 172.21.15.107:0/1408813489 >> 172.21.15.107:0/1408813489 conn(0x7f706c0fe740 msgr2=0x7f706c100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 -- 172.21.15.107:0/1408813489 shutdown_connections 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.079+0000 7f70720b8640 1 -- 172.21.15.107:0/1408813489 wait complete. 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 Processor -- start 2024-09-21T12:09:36.082 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 -- start start 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 0x7f706c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c108700 0x7f706c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 0x7f706c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f706c1121a0 con 0x7f706c109080 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f706c112020 con 0x7f706c108700 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f70720b8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f706c112320 con 0x7f706c102d00 2024-09-21T12:09:36.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 0x7f706c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 0x7f706c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 0x7f706c0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:33316/0 (socket says 172.21.15.107:33316) 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 0x7f706c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42568/0 (socket says 172.21.15.107:42568) 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 -- 172.21.15.107:0/3104584286 learned_addr learned my addr 172.21.15.107:0/3104584286 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706affd640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c108700 0x7f706c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 -- 172.21.15.107:0/3104584286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c108700 msgr2=0x7f706c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c108700 0x7f706c0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 -- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 msgr2=0x7f706c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 0x7f706c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706b7fe640 1 -- 172.21.15.107:0/3104584286 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c1a6b60 con 0x7f706c102d00 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.080+0000 7f706bfff640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 0x7f706c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.081+0000 7f706b7fe640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 0x7f706c0754a0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f705400e9c0 tx=0x7f705400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:09:36.085 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.081+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705400cde0 con 0x7f706c102d00 2024-09-21T12:09:36.086 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.081+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7054004470 con 0x7f706c102d00 2024-09-21T12:09:36.086 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.081+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054003e50 con 0x7f706c102d00 2024-09-21T12:09:36.086 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.081+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c1a6e50 con 0x7f706c102d00 2024-09-21T12:09:36.086 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.081+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c1a72c0 con 0x7f706c102d00 2024-09-21T12:09:36.086 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.082+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706c1030e0 con 0x7f706c102d00 2024-09-21T12:09:36.086 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.082+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f7054004000 con 0x7f706c102d00 2024-09-21T12:09:36.087 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.083+0000 7f7068ff9640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f705807b2f0 0x7f705807d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:09:36.087 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.083+0000 7f706affd640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f705807b2f0 0x7f705807d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:09:36.087 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.083+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(743..743 src has 1..743) ==== 9050+0+0 (secure 0 0 0) 0x7f705409f1f0 con 0x7f706c102d00 2024-09-21T12:09:36.087 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.083+0000 7f706affd640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f705807b2f0 0x7f705807d7b0 secure :-1 s=READY pgs=3929 cs=0 l=1 rev1=1 crypto rx=0x7f705c005940 tx=0x7f705c0058b0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:09:36.089 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.087+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7054069c50 con 0x7f706c102d00 2024-09-21T12:09:36.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:09:36.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[30293]: osdmap e742: 8 total, 8 up, 8 in 2024-09-21T12:09:36.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3163807476' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:36.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:09:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[41097]: osdmap e742: 8 total, 8 up, 8 in 2024-09-21T12:09:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3163807476' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:36.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:35 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:36.270 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.269+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} v 0) -- 0x7f706c1a76b0 con 0x7f706c102d00 2024-09-21T12:09:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:09:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:35 smithi193 ceph-mon[33333]: osdmap e742: 8 total, 8 up, 8 in 2024-09-21T12:09:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:35 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3163807476' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:36.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:35 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:09:36.980 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:36.978+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]=0 enabled application 'rados' on pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' v744) ==== 213+0+0 (secure 0 0 0) 0x7f705406eb00 con 0x7f706c102d00 2024-09-21T12:09:37.094 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.092+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} v 0) -- 0x7f706c1a7900 con 0x7f706c102d00 2024-09-21T12:09:37.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[30293]: pgmap v1145: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:09:37.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[30293]: osdmap e743: 8 total, 8 up, 8 in 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3104584286' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[41097]: pgmap v1145: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[41097]: osdmap e743: 8 total, 8 up, 8 in 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3104584286' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:37.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:36 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:36 smithi193 ceph-mon[33333]: pgmap v1145: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:09:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:09:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:36 smithi193 ceph-mon[33333]: osdmap e743: 8 total, 8 up, 8 in 2024-09-21T12:09:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:36 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3104584286' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:36 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:37.993 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.991+0000 7f7068ff9640 1 -- 172.21.15.107:0/3104584286 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]=0 enabled application 'rados' on pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' v745) ==== 213+0+0 (secure 0 0 0) 0x7f70540035f0 con 0x7f706c102d00 2024-09-21T12:09:37.993 INFO:tasks.workunit.client.0.smithi107.stderr:enabled application 'rados' on pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' 2024-09-21T12:09:37.996 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f705807b2f0 msgr2=0x7f705807d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:37.996 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f705807b2f0 0x7f705807d7b0 secure :-1 s=READY pgs=3929 cs=0 l=1 rev1=1 crypto rx=0x7f705c005940 tx=0x7f705c0058b0 comp rx=0 tx=0).stop 2024-09-21T12:09:37.996 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 msgr2=0x7f706c0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:09:37.996 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 0x7f706c0754a0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f705400e9c0 tx=0x7f705400ee80 comp rx=0 tx=0).stop 2024-09-21T12:09:37.996 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 shutdown_connections 2024-09-21T12:09:37.996 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f705807b2f0 0x7f705807d7b0 unknown :-1 s=CLOSED pgs=3929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:37.997 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f706c109080 0x7f706c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:37.997 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c108700 0x7f706c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:37.997 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 --2- 172.21.15.107:0/3104584286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f706c102d00 0x7f706c0754a0 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:09:37.997 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 >> 172.21.15.107:0/3104584286 conn(0x7f706c0fe740 msgr2=0x7f706c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:09:37.997 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 shutdown_connections 2024-09-21T12:09:37.997 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:09:37.994+0000 7f70720b8640 1 -- 172.21.15.107:0/3104584286 wait complete. 2024-09-21T12:09:38.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ seq 1 10 2024-09-21T12:09:38.024 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.024 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj1 /etc/passwd 2024-09-21T12:09:38.068 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.068 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj2 /etc/passwd 2024-09-21T12:09:38.108 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.108 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj3 /etc/passwd 2024-09-21T12:09:38.144 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.144 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj4 /etc/passwd 2024-09-21T12:09:38.180 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.180 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj5 /etc/passwd 2024-09-21T12:09:38.215 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.216 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj6 /etc/passwd 2024-09-21T12:09:38.251 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.251 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj7 /etc/passwd 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]': finished 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[41097]: osdmap e744: 8 total, 8 up, 8 in 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3104584286' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[41097]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]': finished 2024-09-21T12:09:38.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[30293]: osdmap e744: 8 total, 8 up, 8 in 2024-09-21T12:09:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3104584286' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:38.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:37 smithi107 ceph-mon[30293]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:38.287 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.287 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj8 /etc/passwd 2024-09-21T12:09:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]': finished 2024-09-21T12:09:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:37 smithi193 ceph-mon[33333]: osdmap e744: 8 total, 8 up, 8 in 2024-09-21T12:09:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:37 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3104584286' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:37 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"} : dispatch 2024-09-21T12:09:38.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:37 smithi193 ceph-mon[33333]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:38.324 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.324 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj9 /etc/passwd 2024-09-21T12:09:38.360 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:09:38.360 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put obj10 /etc/passwd 2024-09-21T12:09:38.395 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-21T12:09:39.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[30293]: pgmap v1148: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]': finished 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[30293]: osdmap e745: 8 total, 8 up, 8 in 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[41097]: pgmap v1148: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]': finished 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[41097]: osdmap e745: 8 total, 8 up, 8 in 2024-09-21T12:09:39.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:38 smithi193 ceph-mon[33333]: pgmap v1148: 176 pgs: 12 unknown, 164 active+clean; 583 KiB data, 822 MiB used, 714 GiB / 715 GiB avail 2024-09-21T12:09:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:38 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "app": "rados"}]': finished 2024-09-21T12:09:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:38 smithi193 ceph-mon[33333]: osdmap e745: 8 total, 8 up, 8 in 2024-09-21T12:09:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:39.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:09:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:09:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:40 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:41.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:40 smithi107 ceph-mon[30293]: pgmap v1150: 176 pgs: 176 active+clean; 708 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 3 op/s 2024-09-21T12:09:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:40 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:41.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:40 smithi107 ceph-mon[41097]: pgmap v1150: 176 pgs: 176 active+clean; 708 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 3 op/s 2024-09-21T12:09:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:40 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:40 smithi193 ceph-mon[33333]: pgmap v1150: 176 pgs: 176 active+clean; 708 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 3 op/s 2024-09-21T12:09:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:09:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:09:43.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[30293]: pgmap v1151: 176 pgs: 176 active+clean; 708 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[30293]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[30293]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (reached quota's max_objects: 10) 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[30293]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[30293]: osdmap e746: 8 total, 8 up, 8 in 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[41097]: pgmap v1151: 176 pgs: 176 active+clean; 708 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[41097]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[41097]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (reached quota's max_objects: 10) 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[41097]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:09:43.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:42 smithi107 ceph-mon[41097]: osdmap e746: 8 total, 8 up, 8 in 2024-09-21T12:09:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:42 smithi193 ceph-mon[33333]: pgmap v1151: 176 pgs: 176 active+clean; 708 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-21T12:09:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:42 smithi193 ceph-mon[33333]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T12:09:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:42 smithi193 ceph-mon[33333]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (reached quota's max_objects: 10) 2024-09-21T12:09:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:42 smithi193 ceph-mon[33333]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:09:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:42 smithi193 ceph-mon[33333]: osdmap e746: 8 total, 8 up, 8 in 2024-09-21T12:09:45.000 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:09:44 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=sqlstore.transactions t=2024-09-21T12:09:44.593297789Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-21T12:09:45.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:44 smithi107 ceph-mon[30293]: pgmap v1153: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-21T12:09:45.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:44 smithi107 ceph-mon[41097]: pgmap v1153: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-21T12:09:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:44 smithi193 ceph-mon[33333]: pgmap v1153: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-21T12:09:47.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:47 smithi107 ceph-mon[30293]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-21T12:09:47.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:47 smithi107 ceph-mon[41097]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-21T12:09:47.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:47 smithi193 ceph-mon[33333]: pgmap v1154: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-21T12:09:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:49 smithi193 ceph-mon[33333]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-21T12:09:49.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:49 smithi107 ceph-mon[30293]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-21T12:09:49.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:49 smithi107 ceph-mon[41097]: pgmap v1155: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-21T12:09:49.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:09:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:09:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:51 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:51 smithi193 ceph-mon[33333]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:09:51.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:51 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:51.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:51 smithi107 ceph-mon[30293]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:09:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:51 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:09:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:51 smithi107 ceph-mon[41097]: pgmap v1156: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:09:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:09:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:09:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:09:53.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:53 smithi193 ceph-mon[33333]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:09:53.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:53 smithi107 ceph-mon[41097]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:09:53.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:53 smithi107 ceph-mon[30293]: pgmap v1157: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:09:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:54 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:54 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:54.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:54 smithi193 ceph-mon[33333]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:09:55.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:54 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:55.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:54 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:55.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:54 smithi107 ceph-mon[41097]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:09:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:54 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:09:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:54 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:09:55.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:54 smithi107 ceph-mon[30293]: pgmap v1158: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:09:57.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:56 smithi107 ceph-mon[30293]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:09:57.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:56 smithi107 ceph-mon[41097]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:09:57.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:56 smithi193 ceph-mon[33333]: pgmap v1159: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:09:58.753 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:09:58 smithi107 ceph-mon[30293]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:09:58.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:09:58 smithi107 ceph-mon[41097]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:09:58.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:09:58 smithi193 ceph-mon[33333]: pgmap v1160: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:09:59.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:09:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:10:01.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[30293]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (running out of quota) 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-21T12:10:01.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-21T12:10:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-21T12:10:01.005 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:00 smithi107 ceph-mon[41097]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (running out of quota) 2024-09-21T12:10:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: pgmap v1161: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:01.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-21T12:10:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-21T12:10:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-21T12:10:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-21T12:10:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-21T12:10:01.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:00 smithi193 ceph-mon[33333]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (running out of quota) 2024-09-21T12:10:02.630 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:10:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:10:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:10:03.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:02 smithi107 ceph-mon[30293]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:03.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:02 smithi107 ceph-mon[41097]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:03.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:02 smithi193 ceph-mon[33333]: pgmap v1162: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:05.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:04 smithi107 ceph-mon[30293]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:05.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:04 smithi107 ceph-mon[41097]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:05.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:04 smithi193 ceph-mon[33333]: pgmap v1163: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:07.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:06 smithi107 ceph-mon[30293]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:07.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:06 smithi107 ceph-mon[41097]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:07.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:06 smithi193 ceph-mon[33333]: pgmap v1164: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:08.396 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=147686 2024-09-21T12:10:08.397 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_objects 100 2024-09-21T12:10:08.397 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put onemore /etc/passwd 2024-09-21T12:10:08.495 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 -- 172.21.15.107:0/351647550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 msgr2=0x7f2fc810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:08.495 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 --2- 172.21.15.107:0/351647550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc810e770 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009900 tx=0x7f2fc002a410 comp rx=0 tx=0).stop 2024-09-21T12:10:08.495 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 -- 172.21.15.107:0/351647550 shutdown_connections 2024-09-21T12:10:08.495 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 --2- 172.21.15.107:0/351647550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc810e770 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:08.495 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 --2- 172.21.15.107:0/351647550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 --2- 172.21.15.107:0/351647550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 0x7f2fc8101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 -- 172.21.15.107:0/351647550 >> 172.21.15.107:0/351647550 conn(0x7f2fc80fd650 msgr2=0x7f2fc80ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 -- 172.21.15.107:0/351647550 shutdown_connections 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.491+0000 7f2fd0223640 1 -- 172.21.15.107:0/351647550 wait complete. 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 Processor -- start 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 -- start start 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 0x7f2fc8197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8111100 con 0x7f2fc81075f0 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8110f80 con 0x7f2fc8101bf0 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fd0223640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc8111280 con 0x7f2fc8107f70 2024-09-21T12:10:08.496 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fce7c0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fcd7be640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fce7c0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:50372/0 (socket says 172.21.15.107:50372) 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fce7c0640 1 -- 172.21.15.107:0/1157101663 learned_addr learned my addr 172.21.15.107:0/1157101663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fcd7be640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59026/0 (socket says 172.21.15.107:59026) 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.492+0000 7f2fcdfbf640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 0x7f2fc8197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcd7be640 1 -- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 msgr2=0x7f2fc819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcd7be640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcd7be640 1 -- 172.21.15.107:0/1157101663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 msgr2=0x7f2fc8197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcd7be640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 0x7f2fc8197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcd7be640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc819d6c0 con 0x7f2fc81075f0 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcdfbf640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 0x7f2fc8197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fce7c0640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fcd7be640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8197b60 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e9f0 tx=0x7f2fbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:08.497 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00cd80 con 0x7f2fc81075f0 2024-09-21T12:10:08.498 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc819d9b0 con 0x7f2fc81075f0 2024-09-21T12:10:08.498 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f2fbc004470 con 0x7f2fc81075f0 2024-09-21T12:10:08.498 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc003e10 con 0x7f2fc81075f0 2024-09-21T12:10:08.498 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.493+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc81a4250 con 0x7f2fc81075f0 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.494+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc8101fd0 con 0x7f2fc81075f0 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.495+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f2fbc003fb0 con 0x7f2fc81075f0 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.495+0000 7f2fc6ffd640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f2fb007b150 0x7f2fb007d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.495+0000 7f2fcdfbf640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f2fb007b150 0x7f2fb007d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.495+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(746..746 src has 1..746) ==== 9063+0+0 (secure 0 0 0) 0x7f2fbc09f180 con 0x7f2fc81075f0 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.495+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=747}) -- 0x7f2fb0086bd0 con 0x7f2fc81075f0 2024-09-21T12:10:08.499 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.496+0000 7f2fcdfbf640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f2fb007b150 0x7f2fb007d610 secure :-1 s=READY pgs=3941 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8005fd0 tx=0x7f2fb8009500 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:08.501 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.500+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fbc069bd0 con 0x7f2fc81075f0 2024-09-21T12:10:08.684 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:08.682+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} v 0) -- 0x7f2fc8109d50 con 0x7f2fc81075f0 2024-09-21T12:10:09.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:08 smithi107 ceph-mon[41097]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:09.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:08 smithi107 ceph-mon[30293]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:09.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:08 smithi193 ceph-mon[33333]: pgmap v1165: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:09.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:09.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:10:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:10:09.645 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:09.643+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v747) ==== 225+0+0 (secure 0 0 0) 0x7f2fbc06ea80 con 0x7f2fc81075f0 2024-09-21T12:10:09.645 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:09.643+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 8 ==== osd_map(747..747 src has 1..747) ==== 628+0+0 (secure 0 0 0) 0x7f2fbc0d6710 con 0x7f2fc81075f0 2024-09-21T12:10:09.645 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:09.643+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=748}) -- 0x7f2fb0087b30 con 0x7f2fc81075f0 2024-09-21T12:10:09.760 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:09.758+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} v 0) -- 0x7f2fc81097c0 con 0x7f2fc81075f0 2024-09-21T12:10:10.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:09 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} : dispatch 2024-09-21T12:10:10.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:09 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} : dispatch 2024-09-21T12:10:10.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:09 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} : dispatch 2024-09-21T12:10:10.646 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.645+0000 7f2fc6ffd640 1 -- 172.21.15.107:0/1157101663 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v748) ==== 225+0+0 (secure 0 0 0) 0x7f2fbc009420 con 0x7f2fc81075f0 2024-09-21T12:10:10.647 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 100 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f2fb007b150 msgr2=0x7f2fb007d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f2fb007b150 0x7f2fb007d610 secure :-1 s=READY pgs=3941 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8005fd0 tx=0x7f2fb8009500 comp rx=0 tx=0).stop 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 msgr2=0x7f2fc8197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8197b60 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e9f0 tx=0x7f2fbc00eeb0 comp rx=0 tx=0).stop 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 shutdown_connections 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f2fb007b150 0x7f2fb007d610 unknown :-1 s=CLOSED pgs=3941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc8107f70 0x7f2fc819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc81075f0 0x7f2fc8197b60 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 --2- 172.21.15.107:0/1157101663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fc8101bf0 0x7f2fc8197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:10.650 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 >> 172.21.15.107:0/1157101663 conn(0x7f2fc80fd650 msgr2=0x7f2fc80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:10.651 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 shutdown_connections 2024-09-21T12:10:10.651 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:10.648+0000 7f2fd0223640 1 -- 172.21.15.107:0/1157101663 wait complete. 2024-09-21T12:10:10.665 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 147686 2024-09-21T12:10:11.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:11.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[41097]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]': finished 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[41097]: osdmap e747: 8 total, 8 up, 8 in 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} : dispatch 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[30293]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]': finished 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[30293]: osdmap e747: 8 total, 8 up, 8 in 2024-09-21T12:10:11.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:10 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} : dispatch 2024-09-21T12:10:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:10 smithi193 ceph-mon[33333]: pgmap v1166: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]': finished 2024-09-21T12:10:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:10 smithi193 ceph-mon[33333]: osdmap e747: 8 total, 8 up, 8 in 2024-09-21T12:10:11.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:10 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"} : dispatch 2024-09-21T12:10:12.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:11 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]': finished 2024-09-21T12:10:12.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:11 smithi107 ceph-mon[30293]: osdmap e748: 8 total, 8 up, 8 in 2024-09-21T12:10:12.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:11 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]': finished 2024-09-21T12:10:12.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:11 smithi107 ceph-mon[41097]: osdmap e748: 8 total, 8 up, 8 in 2024-09-21T12:10:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:11 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1157101663' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "100"}]': finished 2024-09-21T12:10:12.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:11 smithi193 ceph-mon[33333]: osdmap e748: 8 total, 8 up, 8 in 2024-09-21T12:10:12.454 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 0 -ne 0 ']' 2024-09-21T12:10:12.454 INFO:tasks.workunit.client.0.smithi107.stderr:+ true 2024-09-21T12:10:12.454 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put twomore /etc/passwd 2024-09-21T12:10:12.493 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_bytes 100 2024-09-21T12:10:12.580 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.576+0000 7f52d26fe640 1 -- 172.21.15.107:0/1532327956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc1043b0 msgr2=0x7f52cc104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.576+0000 7f52d26fe640 1 --2- 172.21.15.107:0/1532327956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc1043b0 0x7f52cc104810 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009900 tx=0x7f52c002a410 comp rx=0 tx=0).stop 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 -- 172.21.15.107:0/1532327956 shutdown_connections 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 --2- 172.21.15.107:0/1532327956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc104d50 0x7f52cc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 --2- 172.21.15.107:0/1532327956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc1043b0 0x7f52cc104810 unknown :-1 s=CLOSED pgs=3022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 --2- 172.21.15.107:0/1532327956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc101f90 0x7f52cc102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 -- 172.21.15.107:0/1532327956 >> 172.21.15.107:0/1532327956 conn(0x7f52cc0776a0 msgr2=0x7f52cc077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 -- 172.21.15.107:0/1532327956 shutdown_connections 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 -- 172.21.15.107:0/1532327956 wait complete. 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 Processor -- start 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 -- start start 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 0x7f52cc197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.577+0000 7f52d26fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc1043b0 0x7f52cc197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:12.581 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d26fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 0x7f52cc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d26fe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1110e0 con 0x7f52cc104d50 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d26fe640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc110f60 con 0x7f52cc1043b0 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d26fe640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc111260 con 0x7f52cc101f90 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 0x7f52cc197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 0x7f52cc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 0x7f52cc197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59076/0 (socket says 172.21.15.107:59076) 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 0x7f52cc197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:50406/0 (socket says 172.21.15.107:50406) 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 -- 172.21.15.107:0/2461513147 learned_addr learned my addr 172.21.15.107:0/2461513147 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52cb7fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc1043b0 0x7f52cc197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 -- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 msgr2=0x7f52cc197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 0x7f52cc197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 -- 172.21.15.107:0/2461513147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc1043b0 msgr2=0x7f52cc197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc1043b0 0x7f52cc197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:12.582 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 -- 172.21.15.107:0/2461513147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc1a6b50 con 0x7f52cc104d50 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52cbfff640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 0x7f52cc197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52cb7fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc1043b0 0x7f52cc197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d0c9b640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 0x7f52cc197dc0 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7f52c400eb10 tx=0x7f52c400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c400ce20 con 0x7f52cc104d50 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f52c4005020 con 0x7f52cc104d50 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc1a6e40 con 0x7f52cc104d50 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4010690 con 0x7f52cc104d50 2024-09-21T12:10:12.583 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.578+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1a73b0 con 0x7f52cc104d50 2024-09-21T12:10:12.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.579+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc102370 con 0x7f52cc104d50 2024-09-21T12:10:12.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.580+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f52c4004a40 con 0x7f52cc104d50 2024-09-21T12:10:12.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.581+0000 7f52c97fa640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f52b407b150 0x7f52b407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:12.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.581+0000 7f52cbfff640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f52b407b150 0x7f52b407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:12.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.581+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(749..749 src has 1..749) ==== 9063+0+0 (secure 0 0 0) 0x7f52c409f370 con 0x7f52cc104d50 2024-09-21T12:10:12.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.581+0000 7f52cbfff640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f52b407b150 0x7f52b407d610 secure :-1 s=READY pgs=3943 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00afb0 tx=0x7f52bc00a2d0 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:12.587 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.585+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52c4069e40 con 0x7f52cc104d50 2024-09-21T12:10:12.666 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:10:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:10:12.769 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:12.768+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} v 0) -- 0x7f52cc1097a0 con 0x7f52cc104d50 2024-09-21T12:10:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[30293]: pgmap v1169: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:10:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[30293]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' no longer out of quota; removing NO_QUOTA flag 2024-09-21T12:10:13.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[30293]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:10:13.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[30293]: osdmap e749: 8 total, 8 up, 8 in 2024-09-21T12:10:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[41097]: pgmap v1169: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:10:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[41097]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' no longer out of quota; removing NO_QUOTA flag 2024-09-21T12:10:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[41097]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:10:13.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:12 smithi107 ceph-mon[41097]: osdmap e749: 8 total, 8 up, 8 in 2024-09-21T12:10:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:12 smithi193 ceph-mon[33333]: pgmap v1169: 176 pgs: 176 active+clean; 708 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T12:10:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:12 smithi193 ceph-mon[33333]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' no longer out of quota; removing NO_QUOTA flag 2024-09-21T12:10:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:12 smithi193 ceph-mon[33333]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:10:13.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:12 smithi193 ceph-mon[33333]: osdmap e749: 8 total, 8 up, 8 in 2024-09-21T12:10:13.673 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:13.671+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v750) ==== 221+0+0 (secure 0 0 0) 0x7f52c406ecf0 con 0x7f52cc104d50 2024-09-21T12:10:13.786 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:13.784+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} v 0) -- 0x7f52cc1099f0 con 0x7f52cc104d50 2024-09-21T12:10:14.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:13 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} : dispatch 2024-09-21T12:10:14.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:13 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} : dispatch 2024-09-21T12:10:14.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:13 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} : dispatch 2024-09-21T12:10:14.674 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.673+0000 7f52c97fa640 1 -- 172.21.15.107:0/2461513147 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v751) ==== 221+0+0 (secure 0 0 0) 0x7f52c409cc10 con 0x7f52cc104d50 2024-09-21T12:10:14.675 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_bytes = 100 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:10:14.679 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f52b407b150 msgr2=0x7f52b407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:14.679 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f52b407b150 0x7f52b407d610 secure :-1 s=READY pgs=3943 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00afb0 tx=0x7f52bc00a2d0 comp rx=0 tx=0).stop 2024-09-21T12:10:14.679 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 msgr2=0x7f52cc197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:14.679 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 0x7f52cc197dc0 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7f52c400eb10 tx=0x7f52c400ef10 comp rx=0 tx=0).stop 2024-09-21T12:10:14.679 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 shutdown_connections 2024-09-21T12:10:14.679 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f52b407b150 0x7f52b407d610 unknown :-1 s=CLOSED pgs=3943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:14.680 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52cc104d50 0x7f52cc197dc0 unknown :-1 s=CLOSED pgs=3023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:14.680 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52cc1043b0 0x7f52cc197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:14.680 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 --2- 172.21.15.107:0/2461513147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc101f90 0x7f52cc197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:14.680 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.677+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 >> 172.21.15.107:0/2461513147 conn(0x7f52cc0776a0 msgr2=0x7f52cc105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:14.680 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.678+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 shutdown_connections 2024-09-21T12:10:14.680 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:14.678+0000 7f52d26fe640 1 -- 172.21.15.107:0/2461513147 wait complete. 2024-09-21T12:10:14.691 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-21T12:10:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[30293]: pgmap v1171: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-21T12:10:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]': finished 2024-09-21T12:10:15.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[30293]: osdmap e750: 8 total, 8 up, 8 in 2024-09-21T12:10:15.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} : dispatch 2024-09-21T12:10:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[41097]: pgmap v1171: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-21T12:10:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]': finished 2024-09-21T12:10:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[41097]: osdmap e750: 8 total, 8 up, 8 in 2024-09-21T12:10:15.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:14 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} : dispatch 2024-09-21T12:10:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:14 smithi193 ceph-mon[33333]: pgmap v1171: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-21T12:10:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]': finished 2024-09-21T12:10:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:14 smithi193 ceph-mon[33333]: osdmap e750: 8 total, 8 up, 8 in 2024-09-21T12:10:15.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:14 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"} : dispatch 2024-09-21T12:10:16.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:15 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]': finished 2024-09-21T12:10:16.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:15 smithi107 ceph-mon[30293]: osdmap e751: 8 total, 8 up, 8 in 2024-09-21T12:10:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:15 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]': finished 2024-09-21T12:10:16.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:15 smithi107 ceph-mon[41097]: osdmap e751: 8 total, 8 up, 8 in 2024-09-21T12:10:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:15 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2461513147' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "100"}]': finished 2024-09-21T12:10:16.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:15 smithi193 ceph-mon[33333]: osdmap e751: 8 total, 8 up, 8 in 2024-09-21T12:10:17.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:16 smithi107 ceph-mon[30293]: pgmap v1174: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-21T12:10:17.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:16 smithi107 ceph-mon[41097]: pgmap v1174: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-21T12:10:17.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:16 smithi193 ceph-mon[33333]: pgmap v1174: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-21T12:10:18.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:17 smithi107 ceph-mon[30293]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (reached quota's max_bytes: 100 B) 2024-09-21T12:10:18.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:17 smithi107 ceph-mon[30293]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:10:18.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:17 smithi107 ceph-mon[30293]: osdmap e752: 8 total, 8 up, 8 in 2024-09-21T12:10:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:17 smithi107 ceph-mon[41097]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (reached quota's max_bytes: 100 B) 2024-09-21T12:10:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:17 smithi107 ceph-mon[41097]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:10:18.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:17 smithi107 ceph-mon[41097]: osdmap e752: 8 total, 8 up, 8 in 2024-09-21T12:10:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:17 smithi193 ceph-mon[33333]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' is full (reached quota's max_bytes: 100 B) 2024-09-21T12:10:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:17 smithi193 ceph-mon[33333]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:10:18.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:17 smithi193 ceph-mon[33333]: osdmap e752: 8 total, 8 up, 8 in 2024-09-21T12:10:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:18 smithi107 ceph-mon[30293]: pgmap v1176: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-21T12:10:19.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:18 smithi107 ceph-mon[41097]: pgmap v1176: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-21T12:10:19.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:18 smithi193 ceph-mon[33333]: pgmap v1176: 176 pgs: 176 active+clean; 733 KiB data, 823 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-21T12:10:19.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:10:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:10:21.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:20 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:21.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:20 smithi107 ceph-mon[30293]: pgmap v1177: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:20 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:21.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:20 smithi107 ceph-mon[41097]: pgmap v1177: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:20 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:21.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:20 smithi193 ceph-mon[33333]: pgmap v1177: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:22.698 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:10:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:10:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:10:23.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:22 smithi107 ceph-mon[30293]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 647 B/s rd, 0 op/s 2024-09-21T12:10:23.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:22 smithi107 ceph-mon[41097]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 647 B/s rd, 0 op/s 2024-09-21T12:10:23.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:22 smithi193 ceph-mon[33333]: pgmap v1178: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 647 B/s rd, 0 op/s 2024-09-21T12:10:24.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:24.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:24.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:25.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:24 smithi107 ceph-mon[30293]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:10:25.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:24 smithi107 ceph-mon[41097]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:10:25.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:24 smithi193 ceph-mon[33333]: pgmap v1179: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:10:27.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:26 smithi107 ceph-mon[30293]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:10:27.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:26 smithi107 ceph-mon[41097]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:10:27.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:26 smithi193 ceph-mon[33333]: pgmap v1180: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:10:29.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:28 smithi107 ceph-mon[30293]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-21T12:10:29.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:28 smithi107 ceph-mon[41097]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-21T12:10:29.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:28 smithi193 ceph-mon[33333]: pgmap v1181: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-21T12:10:29.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:10:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:10:31.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:30 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:31.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:30 smithi107 ceph-mon[30293]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:30 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:31.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:30 smithi107 ceph-mon[41097]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:30 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:31.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:30 smithi193 ceph-mon[33333]: pgmap v1182: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:32.721 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:10:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:10:33.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[30293]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:33.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[41097]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:10:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:32 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:10:33.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:32 smithi193 ceph-mon[33333]: pgmap v1183: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:33.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:32 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:10:33.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:32 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:10:33.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:32 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:10:33.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:32 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:10:35.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:34 smithi107 ceph-mon[30293]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:35.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:34 smithi107 ceph-mon[41097]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:35.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:34 smithi193 ceph-mon[33333]: pgmap v1184: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:37.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:36 smithi107 ceph-mon[30293]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:37.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:36 smithi107 ceph-mon[41097]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:37.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:36 smithi193 ceph-mon[33333]: pgmap v1185: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:39.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:38 smithi107 ceph-mon[30293]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:39.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:38 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:38 smithi107 ceph-mon[41097]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:39.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:38 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:38 smithi193 ceph-mon[33333]: pgmap v1186: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:39.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:38 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:39.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:10:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:10:41.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:40 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:41.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:40 smithi107 ceph-mon[30293]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:40 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:41.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:40 smithi107 ceph-mon[41097]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:40 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:41.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:40 smithi193 ceph-mon[33333]: pgmap v1187: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:42.735 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:10:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:10:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:10:43.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:42 smithi107 ceph-mon[30293]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:10:43.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:42 smithi107 ceph-mon[41097]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:10:43.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:42 smithi193 ceph-mon[33333]: pgmap v1188: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:10:44.692 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=147856 2024-09-21T12:10:44.693 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_bytes 0 2024-09-21T12:10:44.693 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put two /etc/passwd 2024-09-21T12:10:44.790 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 -- 172.21.15.107:0/809102918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f0109070 msgr2=0x7fb2f010f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/809102918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f0109070 0x7fb2f010f870 secure :-1 s=READY pgs=3027 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009ca0 tx=0x7fb2e802a8c0 comp rx=0 tx=0).stop 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 -- 172.21.15.107:0/809102918 shutdown_connections 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/809102918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f0109070 0x7fb2f010f870 unknown :-1 s=CLOSED pgs=3027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/809102918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f01086f0 0x7fb2f0108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/809102918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2f0102cf0 0x7fb2f01030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 -- 172.21.15.107:0/809102918 >> 172.21.15.107:0/809102918 conn(0x7fb2f00fe750 msgr2=0x7fb2f0100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 -- 172.21.15.107:0/809102918 shutdown_connections 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.788+0000 7fb2f68ef640 1 -- 172.21.15.107:0/809102918 wait complete. 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 Processor -- start 2024-09-21T12:10:44.791 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 -- start start 2024-09-21T12:10:44.793 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 0x7fb2f00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:44.793 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 0x7fb2f00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:44.793 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2f0109070 0x7fb2f0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0112200 con 0x7fb2f01086f0 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.789+0000 7fb2f68ef640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0112080 con 0x7fb2f0109070 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2f68ef640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f0112380 con 0x7fb2f0102cf0 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2effff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 0x7fb2f00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 0x7fb2f00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2effff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 0x7fb2f00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41302/0 (socket says 172.21.15.107:41302) 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2effff640 1 -- 172.21.15.107:0/523736155 learned_addr learned my addr 172.21.15.107:0/523736155 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 0x7fb2f00759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54890/0 (socket says 172.21.15.107:54890) 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2f4e8c640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2f0109070 0x7fb2f0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 -- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 msgr2=0x7fb2f00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 0x7fb2f00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 -- 172.21.15.107:0/523736155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2f0109070 msgr2=0x7fb2f0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:44.794 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2f0109070 0x7fb2f0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f01a6a70 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2effff640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 0x7fb2f00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ef7fe640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 0x7fb2f00759e0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e9f0 tx=0x7fb2e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400cd80 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fb2e4004470 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a6d60 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4003e10 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.790+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a7250 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.791+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f01030d0 con 0x7fb2f01086f0 2024-09-21T12:10:44.795 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.792+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7fb2e4003fb0 con 0x7fb2f01086f0 2024-09-21T12:10:44.796 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.792+0000 7fb2ed7fa640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fb2d807b150 0x7fb2d807d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:44.796 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.793+0000 7fb2effff640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fb2d807b150 0x7fb2d807d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:44.796 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.793+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(752..752 src has 1..752) ==== 9063+0+0 (secure 0 0 0) 0x7fb2e409f1e0 con 0x7fb2f01086f0 2024-09-21T12:10:44.796 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.793+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=753}) -- 0x7fb2d8086bd0 con 0x7fb2f01086f0 2024-09-21T12:10:44.796 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.793+0000 7fb2effff640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fb2d807b150 0x7fb2d807d610 secure :-1 s=READY pgs=3945 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000f4d0 tx=0x7fb2e000a380 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:44.798 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.797+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb2e4069c30 con 0x7fb2f01086f0 2024-09-21T12:10:44.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:44.976+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} v 0) -- 0x7fb2f010a8c0 con 0x7fb2f01086f0 2024-09-21T12:10:45.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:44 smithi107 ceph-mon[30293]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:45.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:44 smithi107 ceph-mon[41097]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:45.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:44 smithi193 ceph-mon[33333]: pgmap v1189: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:10:45.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:45.742+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v753) ==== 217+0+0 (secure 0 0 0) 0x7fb2e406eae0 con 0x7fb2f01086f0 2024-09-21T12:10:45.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:45.743+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 8 ==== osd_map(753..753 src has 1..753) ==== 628+0+0 (secure 0 0 0) 0x7fb2e40d6710 con 0x7fb2f01086f0 2024-09-21T12:10:45.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:45.743+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=754}) -- 0x7fb2d8087b30 con 0x7fb2f01086f0 2024-09-21T12:10:45.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:45.856+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} v 0) -- 0x7fb2f010ab10 con 0x7fb2f01086f0 2024-09-21T12:10:46.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:45 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:10:46.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:45 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:10:46.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:45 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:10:46.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.744+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v754) ==== 217+0+0 (secure 0 0 0) 0x7fb2e4009420 con 0x7fb2f01086f0 2024-09-21T12:10:46.746 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_bytes = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:10:46.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.744+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 <== mon.0 v2:172.21.15.107:3300/0 10 ==== osd_map(754..754 src has 1..754) ==== 628+0+0 (secure 0 0 0) 0x7fb2e409ca80 con 0x7fb2f01086f0 2024-09-21T12:10:46.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.744+0000 7fb2ed7fa640 1 -- 172.21.15.107:0/523736155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=755}) -- 0x7fb2d8087d40 con 0x7fb2f01086f0 2024-09-21T12:10:46.751 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.749+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fb2d807b150 msgr2=0x7fb2d807d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:46.751 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.749+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fb2d807b150 0x7fb2d807d610 secure :-1 s=READY pgs=3945 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000f4d0 tx=0x7fb2e000a380 comp rx=0 tx=0).stop 2024-09-21T12:10:46.751 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.749+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 msgr2=0x7fb2f00759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.749+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 0x7fb2f00759e0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e9f0 tx=0x7fb2e400eeb0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 shutdown_connections 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fb2d807b150 0x7fb2d807d610 unknown :-1 s=CLOSED pgs=3945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2f0109070 0x7fb2f0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f01086f0 0x7fb2f00759e0 unknown :-1 s=CLOSED pgs=3028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 --2- 172.21.15.107:0/523736155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f0102cf0 0x7fb2f00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 >> 172.21.15.107:0/523736155 conn(0x7fb2f00fe750 msgr2=0x7fb2f0100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 shutdown_connections 2024-09-21T12:10:46.752 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.750+0000 7fb2f68ef640 1 -- 172.21.15.107:0/523736155 wait complete. 2024-09-21T12:10:46.765 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_objects 0 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 -- 172.21.15.107:0/3960761033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c4104d50 msgr2=0x7f19c410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/3960761033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c4104d50 0x7f19c410e750 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7f19bc009860 tx=0x7f19bc02a340 comp rx=0 tx=0).stop 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 -- 172.21.15.107:0/3960761033 shutdown_connections 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/3960761033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c4104d50 0x7f19c410e750 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/3960761033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c41043b0 0x7f19c4104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/3960761033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4101f90 0x7f19c4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 -- 172.21.15.107:0/3960761033 >> 172.21.15.107:0/3960761033 conn(0x7f19c40776a0 msgr2=0x7f19c4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:46.852 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 -- 172.21.15.107:0/3960761033 shutdown_connections 2024-09-21T12:10:46.853 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 -- 172.21.15.107:0/3960761033 wait complete. 2024-09-21T12:10:46.853 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 Processor -- start 2024-09-21T12:10:46.853 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.850+0000 7f19c8d5d640 1 -- start start 2024-09-21T12:10:46.855 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 0x7f19c41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 0x7f19c4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4104d50 0x7f19c41a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19c41110e0 con 0x7f19c41043b0 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4110f60 con 0x7f19c4104d50 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f19c4111260 con 0x7f19c4101f90 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c259c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 0x7f19c41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 0x7f19c4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c259c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 0x7f19c41994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41330/0 (socket says 172.21.15.107:41330) 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 0x7f19c4199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54902/0 (socket says 172.21.15.107:54902) 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c259c640 1 -- 172.21.15.107:0/2026936160 learned_addr learned my addr 172.21.15.107:0/2026936160 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:46.856 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c2d9d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4104d50 0x7f19c41a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 -- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 msgr2=0x7f19c41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 0x7f19c41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 -- 172.21.15.107:0/2026936160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4104d50 msgr2=0x7f19c41a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4104d50 0x7f19c41a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c41a4a90 con 0x7f19c41043b0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c2d9d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4104d50 0x7f19c41a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c259c640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 0x7f19c41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c1d9b640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 0x7f19c4199a10 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f19ac005fd0 tx=0x7f19ac00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00cd00 con 0x7f19c41043b0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c41a4c60 con 0x7f19c41043b0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f19ac004470 con 0x7f19c41043b0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac003db0 con 0x7f19c41043b0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.851+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c41a5120 con 0x7f19c41043b0 2024-09-21T12:10:46.857 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.853+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c4102370 con 0x7f19c41043b0 2024-09-21T12:10:46.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.853+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f19ac004000 con 0x7f19c41043b0 2024-09-21T12:10:46.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.854+0000 7f19bb7fe640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f19b007b150 0x7f19b007d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:46.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.854+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(754..754 src has 1..754) ==== 9063+0+0 (secure 0 0 0) 0x7f19ac09f150 con 0x7f19c41043b0 2024-09-21T12:10:46.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.854+0000 7f19c259c640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f19b007b150 0x7f19b007d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:46.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.854+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=755}) -- 0x7f19b0086bd0 con 0x7f19c41043b0 2024-09-21T12:10:46.858 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.854+0000 7f19c259c640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f19b007b150 0x7f19b007d610 secure :-1 s=READY pgs=3946 cs=0 l=1 rev1=1 crypto rx=0x7f19b4007fd0 tx=0x7f19b4007620 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:46.860 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:46.858+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19ac069ba0 con 0x7f19c41043b0 2024-09-21T12:10:47.041 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:47.040+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} v 0) -- 0x7f19c402e5c0 con 0x7f19c41043b0 2024-09-21T12:10:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:46 smithi193 ceph-mon[33333]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:10:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:10:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:46 smithi193 ceph-mon[33333]: osdmap e753: 8 total, 8 up, 8 in 2024-09-21T12:10:47.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:46 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:10:47.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[41097]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:10:47.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:10:47.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[41097]: osdmap e753: 8 total, 8 up, 8 in 2024-09-21T12:10:47.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:10:47.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[30293]: pgmap v1190: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:10:47.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:10:47.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[30293]: osdmap e753: 8 total, 8 up, 8 in 2024-09-21T12:10:47.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:46 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:10:47.449 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:47.447+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v755) ==== 221+0+0 (secure 0 0 0) 0x7f19ac06ea50 con 0x7f19c41043b0 2024-09-21T12:10:47.450 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:47.448+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 8 ==== osd_map(755..755 src has 1..755) ==== 628+0+0 (secure 0 0 0) 0x7f19ac0d6710 con 0x7f19c41043b0 2024-09-21T12:10:47.450 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:47.448+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=756}) -- 0x7f19b0087ad0 con 0x7f19c41043b0 2024-09-21T12:10:47.563 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:47.561+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} v 0) -- 0x7f19c402e810 con 0x7f19c41043b0 2024-09-21T12:10:48.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:10:48.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: osdmap e754: 8 total, 8 up, 8 in 2024-09-21T12:10:48.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:10:48.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' no longer out of quota; removing NO_QUOTA flag 2024-09-21T12:10:48.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:10:48.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:10:48.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: osdmap e755: 8 total, 8 up, 8 in 2024-09-21T12:10:48.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:47 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:10:48.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: osdmap e754: 8 total, 8 up, 8 in 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' no longer out of quota; removing NO_QUOTA flag 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: osdmap e755: 8 total, 8 up, 8 in 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/523736155' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: osdmap e754: 8 total, 8 up, 8 in 2024-09-21T12:10:48.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:10:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' no longer out of quota; removing NO_QUOTA flag 2024-09-21T12:10:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:10:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:10:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: osdmap e755: 8 total, 8 up, 8 in 2024-09-21T12:10:48.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:47 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:10:48.455 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.453+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v756) ==== 221+0+0 (secure 0 0 0) 0x7f19ac06c0b0 con 0x7f19c41043b0 2024-09-21T12:10:48.455 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:10:48.455 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.453+0000 7f19bb7fe640 1 -- 172.21.15.107:0/2026936160 <== mon.0 v2:172.21.15.107:3300/0 10 ==== osd_map(756..756 src has 1..756) ==== 628+0+0 (secure 0 0 0) 0x7f19ac09c9f0 con 0x7f19c41043b0 2024-09-21T12:10:48.459 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.457+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f19b007b150 msgr2=0x7f19b007d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:48.459 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.457+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f19b007b150 0x7f19b007d610 secure :-1 s=READY pgs=3946 cs=0 l=1 rev1=1 crypto rx=0x7f19b4007fd0 tx=0x7f19b4007620 comp rx=0 tx=0).stop 2024-09-21T12:10:48.459 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.458+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 msgr2=0x7f19c4199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:48.459 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.458+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 0x7f19c4199a10 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f19ac005fd0 tx=0x7f19ac00ee50 comp rx=0 tx=0).stop 2024-09-21T12:10:48.460 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.459+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 shutdown_connections 2024-09-21T12:10:48.460 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.459+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f19b007b150 0x7f19b007d610 unknown :-1 s=CLOSED pgs=3946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.461 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.459+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c4104d50 0x7f19c41a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.461 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.459+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19c41043b0 0x7f19c4199a10 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.461 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.459+0000 7f19c8d5d640 1 --2- 172.21.15.107:0/2026936160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19c4101f90 0x7f19c41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.461 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.459+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 >> 172.21.15.107:0/2026936160 conn(0x7f19c40776a0 msgr2=0x7f19c4105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:48.461 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.460+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 shutdown_connections 2024-09-21T12:10:48.462 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.460+0000 7f19c8d5d640 1 -- 172.21.15.107:0/2026936160 wait complete. 2024-09-21T12:10:48.473 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 147856 2024-09-21T12:10:48.473 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 0 -ne 0 ']' 2024-09-21T12:10:48.473 INFO:tasks.workunit.client.0.smithi107.stderr:+ true 2024-09-21T12:10:48.473 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put three /etc/passwd 2024-09-21T12:10:48.512 INFO:tasks.workunit.client.0.smithi107.stderr:++ uuidgen 2024-09-21T12:10:48.513 INFO:tasks.workunit.client.0.smithi107.stderr:+ pp=4a24a534-8178-47ec-9e91-28e33a5c3068 2024-09-21T12:10:48.513 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool create 4a24a534-8178-47ec-9e91-28e33a5c3068 12 2024-09-21T12:10:48.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/794552608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8106cf0 msgr2=0x7fc7b8107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:48.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/794552608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8106cf0 0x7fc7b8107130 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009900 tx=0x7fc7a802a410 comp rx=0 tx=0).stop 2024-09-21T12:10:48.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/794552608 shutdown_connections 2024-09-21T12:10:48.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/794552608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 0x7fc7b810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/794552608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8106cf0 0x7fc7b8107130 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/794552608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7b8101710 0x7fc7b8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/794552608 >> 172.21.15.107:0/794552608 conn(0x7fc7b8076e10 msgr2=0x7fc7b8077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/794552608 shutdown_connections 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.595+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/794552608 wait complete. 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 Processor -- start 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 -- start start 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 0x7fc7b8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7b8106cf0 0x7fc7b8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 0x7fc7b819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810ffe0 con 0x7fc7b8101710 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810fe60 con 0x7fc7b8106cf0 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bfbc9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b8110160 con 0x7fc7b8107670 2024-09-21T12:10:48.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bd965640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 0x7fc7b8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bd965640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 0x7fc7b8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54960/0 (socket says 172.21.15.107:54960) 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7bd965640 1 -- 172.21.15.107:0/74440603 learned_addr learned my addr 172.21.15.107:0/74440603 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.596+0000 7fc7be166640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 0x7fc7b819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7bd164640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7b8106cf0 0x7fc7b8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7be166640 1 -- 172.21.15.107:0/74440603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7b8106cf0 msgr2=0x7fc7b8198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7be166640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7b8106cf0 0x7fc7b8198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7be166640 1 -- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 msgr2=0x7fc7b8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7be166640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 0x7fc7b8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7be166640 1 -- 172.21.15.107:0/74440603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b819e830 con 0x7fc7b8107670 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7bd965640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 0x7fc7b8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7be166640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 0x7fc7b819e060 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000ea10 tx=0x7fc7b000eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000cbc0 con 0x7fc7b8107670 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fc7b0004470 con 0x7fc7b8107670 2024-09-21T12:10:48.601 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00047a0 con 0x7fc7b8107670 2024-09-21T12:10:48.602 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b819eb20 con 0x7fc7b8107670 2024-09-21T12:10:48.602 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.597+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a51e0 con 0x7fc7b8107670 2024-09-21T12:10:48.602 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.598+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8101af0 con 0x7fc7b8107670 2024-09-21T12:10:48.603 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.599+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7fc7b001d020 con 0x7fc7b8107670 2024-09-21T12:10:48.603 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.599+0000 7fc7aeffd640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fc7a407b150 0x7fc7a407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:48.603 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.599+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(756..756 src has 1..756) ==== 9063+0+0 (secure 0 0 0) 0x7fc7b009e8d0 con 0x7fc7b8107670 2024-09-21T12:10:48.603 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.599+0000 7fc7bd965640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fc7a407b150 0x7fc7a407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:48.603 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.599+0000 7fc7bd965640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fc7a407b150 0x7fc7a407d610 secure :-1 s=READY pgs=3948 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009fd0 tx=0x7fc7b4009720 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:48.605 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.603+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc7b0069320 con 0x7fc7b8107670 2024-09-21T12:10:48.786 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:48.785+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} v 0) -- 0x7fc7b81094e0 con 0x7fc7b8107670 2024-09-21T12:10:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:48 smithi193 ceph-mon[33333]: pgmap v1194: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:48 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:10:49.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:48 smithi193 ceph-mon[33333]: osdmap e756: 8 total, 8 up, 8 in 2024-09-21T12:10:49.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:48 smithi107 ceph-mon[41097]: pgmap v1194: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:49.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:48 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:10:49.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:48 smithi107 ceph-mon[41097]: osdmap e756: 8 total, 8 up, 8 in 2024-09-21T12:10:49.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:48 smithi107 ceph-mon[30293]: pgmap v1194: 176 pgs: 176 active+clean; 733 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:10:49.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:48 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2026936160' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:10:49.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:48 smithi107 ceph-mon[30293]: osdmap e756: 8 total, 8 up, 8 in 2024-09-21T12:10:49.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:10:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:10:49.768 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.766+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12}]=0 pool '4a24a534-8178-47ec-9e91-28e33a5c3068' created v757) ==== 176+0+0 (secure 0 0 0) 0x7fc7b006e1d0 con 0x7fc7b8107670 2024-09-21T12:10:49.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.878+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} v 0) -- 0x7fc7b8108f50 con 0x7fc7b8107670 2024-09-21T12:10:49.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.878+0000 7fc7aeffd640 1 -- 172.21.15.107:0/74440603 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12}]=0 pool '4a24a534-8178-47ec-9e91-28e33a5c3068' already exists v757) ==== 183+0+0 (secure 0 0 0) 0x7fc7b0005020 con 0x7fc7b8107670 2024-09-21T12:10:49.880 INFO:tasks.workunit.client.0.smithi107.stderr:pool '4a24a534-8178-47ec-9e91-28e33a5c3068' already exists 2024-09-21T12:10:49.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fc7a407b150 msgr2=0x7fc7a407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fc7a407b150 0x7fc7a407d610 secure :-1 s=READY pgs=3948 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009fd0 tx=0x7fc7b4009720 comp rx=0 tx=0).stop 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 msgr2=0x7fc7b819e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 0x7fc7b819e060 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000ea10 tx=0x7fc7b000eed0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 shutdown_connections 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fc7a407b150 0x7fc7a407d610 unknown :-1 s=CLOSED pgs=3948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7b8107670 0x7fc7b819e060 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7b8106cf0 0x7fc7b8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 --2- 172.21.15.107:0/74440603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7b8101710 0x7fc7b8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 >> 172.21.15.107:0/74440603 conn(0x7fc7b8076e10 msgr2=0x7fc7b8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 shutdown_connections 2024-09-21T12:10:49.883 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.881+0000 7fc7bfbc9640 1 -- 172.21.15.107:0/74440603 wait complete. 2024-09-21T12:10:49.891 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool application enable 4a24a534-8178-47ec-9e91-28e33a5c3068 rados 2024-09-21T12:10:49.974 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 -- 172.21.15.107:0/397297895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8107f60 msgr2=0x7f83f810e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:49.974 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 --2- 172.21.15.107:0/397297895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8107f60 0x7f83f810e760 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009ca0 tx=0x7f83f002a8c0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.974 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 -- 172.21.15.107:0/397297895 shutdown_connections 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 --2- 172.21.15.107:0/397297895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8107f60 0x7f83f810e760 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 --2- 172.21.15.107:0/397297895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 0x7f83f8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 --2- 172.21.15.107:0/397297895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8101be0 0x7f83f8101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 -- 172.21.15.107:0/397297895 >> 172.21.15.107:0/397297895 conn(0x7f83f80fd640 msgr2=0x7f83f80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 -- 172.21.15.107:0/397297895 shutdown_connections 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.972+0000 7f83fdca9640 1 -- 172.21.15.107:0/397297895 wait complete. 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 Processor -- start 2024-09-21T12:10:49.975 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 -- start start 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 0x7f83f8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 0x7f83f8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8107f60 0x7f83f819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81110f0 con 0x7f83f8107f60 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8110f70 con 0x7f83f81075e0 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83fdca9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f83f8111270 con 0x7f83f8101be0 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 0x7f83f8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8107f60 0x7f83f819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:49.977 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 0x7f83f8197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41388/0 (socket says 172.21.15.107:41388) 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 -- 172.21.15.107:0/3585537981 learned_addr learned my addr 172.21.15.107:0/3585537981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83effff640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 0x7f83f8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 -- 172.21.15.107:0/3585537981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 msgr2=0x7f83f8197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 0x7f83f8197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 -- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8107f60 msgr2=0x7f83f819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8107f60 0x7f83f819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f77fe640 1 -- 172.21.15.107:0/3585537981 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f819d7a0 con 0x7f83f8101be0 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83effff640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 0x7f83f8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.973+0000 7f83f7fff640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8107f60 0x7f83f819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.974+0000 7f83f77fe640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 0x7f83f8197700 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 crypto rx=0x7f83e800ea60 tx=0x7f83e800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.974+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800cde0 con 0x7f83f8101be0 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.974+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f83e8004470 con 0x7f83f8101be0 2024-09-21T12:10:49.978 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.974+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f819da90 con 0x7f83f8101be0 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.974+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8010430 con 0x7f83f8101be0 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.974+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81a4150 con 0x7f83f8101be0 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.975+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f8101fc0 con 0x7f83f8101be0 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.975+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f83e8003f60 con 0x7f83f8101be0 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.976+0000 7f83f57fa640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83e407b150 0x7f83e407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.976+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(757..757 src has 1..757) ==== 9438+0+0 (secure 0 0 0) 0x7f83e809f3a0 con 0x7f83f8101be0 2024-09-21T12:10:49.979 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.976+0000 7f83effff640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83e407b150 0x7f83e407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:49.980 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.976+0000 7f83effff640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83e407b150 0x7f83e407d610 secure :-1 s=READY pgs=3949 cs=0 l=1 rev1=1 crypto rx=0x7f83f80fffe0 tx=0x7f83e0008040 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:49.981 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:49.980+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83e8069c80 con 0x7f83f8101be0 2024-09-21T12:10:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:49 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/74440603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:50.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:49 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:50.164 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:50.162+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} v 0) -- 0x7f83f8198820 con 0x7f83f8101be0 2024-09-21T12:10:50.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:49 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/74440603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:50.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:49 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:50.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:49 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/74440603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:50.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:49 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:50.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:50.772+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]=0 enabled application 'rados' on pool '4a24a534-8178-47ec-9e91-28e33a5c3068' v758) ==== 213+0+0 (secure 0 0 0) 0x7f83e806eb30 con 0x7f83f8101be0 2024-09-21T12:10:50.889 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:50.888+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} v 0) -- 0x7f83f81087d0 con 0x7f83f8101be0 2024-09-21T12:10:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: pgmap v1196: 176 pgs: 176 active+clean; 746 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-21T12:10:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12}]': finished 2024-09-21T12:10:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: osdmap e757: 8 total, 8 up, 8 in 2024-09-21T12:10:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/74440603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:51.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:51.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3585537981' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:51.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:50 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:51.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: pgmap v1196: 176 pgs: 176 active+clean; 746 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12}]': finished 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: osdmap e757: 8 total, 8 up, 8 in 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/74440603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3585537981' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: pgmap v1196: 176 pgs: 176 active+clean; 746 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12}]': finished 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: osdmap e757: 8 total, 8 up, 8 in 2024-09-21T12:10:51.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/74440603' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:51.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pg_num": 12} : dispatch 2024-09-21T12:10:51.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3585537981' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:51.255 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:50 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:51.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.773+0000 7f83f57fa640 1 -- 172.21.15.107:0/3585537981 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]=0 enabled application 'rados' on pool '4a24a534-8178-47ec-9e91-28e33a5c3068' v759) ==== 213+0+0 (secure 0 0 0) 0x7f83e8004250 con 0x7f83f8101be0 2024-09-21T12:10:51.775 INFO:tasks.workunit.client.0.smithi107.stderr:enabled application 'rados' on pool '4a24a534-8178-47ec-9e91-28e33a5c3068' 2024-09-21T12:10:51.778 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83e407b150 msgr2=0x7f83e407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:51.778 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83e407b150 0x7f83e407d610 secure :-1 s=READY pgs=3949 cs=0 l=1 rev1=1 crypto rx=0x7f83f80fffe0 tx=0x7f83e0008040 comp rx=0 tx=0).stop 2024-09-21T12:10:51.778 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 msgr2=0x7f83f8197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 0x7f83f8197700 secure :-1 s=READY pgs=3069 cs=0 l=1 rev1=1 crypto rx=0x7f83e800ea60 tx=0x7f83e800ef20 comp rx=0 tx=0).stop 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 shutdown_connections 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f83e407b150 0x7f83e407d610 unknown :-1 s=CLOSED pgs=3949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83f8107f60 0x7f83f819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f81075e0 0x7f83f8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 --2- 172.21.15.107:0/3585537981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83f8101be0 0x7f83f8197700 unknown :-1 s=CLOSED pgs=3069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 >> 172.21.15.107:0/3585537981 conn(0x7f83f80fd640 msgr2=0x7f83f81043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.776+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 shutdown_connections 2024-09-21T12:10:51.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.777+0000 7f83fdca9640 1 -- 172.21.15.107:0/3585537981 wait complete. 2024-09-21T12:10:51.790 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota 4a24a534-8178-47ec-9e91-28e33a5c3068 max_objects 10 2024-09-21T12:10:51.878 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 -- 172.21.15.107:0/2621640189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16810f390 msgr2=0x7fe168111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:51.878 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2621640189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16810f390 0x7fe168111780 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7fe160009900 tx=0x7fe16002a410 comp rx=0 tx=0).stop 2024-09-21T12:10:51.878 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 -- 172.21.15.107:0/2621640189 shutdown_connections 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2621640189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16810f390 0x7fe168111780 unknown :-1 s=CLOSED pgs=3070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2621640189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe16806a810 0x7fe16806ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2621640189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16806a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 -- 172.21.15.107:0/2621640189 >> 172.21.15.107:0/2621640189 conn(0x7fe1680776a0 msgr2=0x7fe168077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 -- 172.21.15.107:0/2621640189 shutdown_connections 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 -- 172.21.15.107:0/2621640189 wait complete. 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 Processor -- start 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 -- start start 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 0x7fe16819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.875+0000 7fe16ff94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16ff94640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe16810f390 0x7fe1681a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16ff94640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1681143a0 con 0x7fe168103890 2024-09-21T12:10:51.879 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16ff94640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe168114220 con 0x7fe16810f390 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16ff94640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7fe168114520 con 0x7fe16806a810 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16dd30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 0x7fe16819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16dd30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 0x7fe16819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41396/0 (socket says 172.21.15.107:41396) 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54980/0 (socket says 172.21.15.107:54980) 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16dd30640 1 -- 172.21.15.107:0/2909223955 learned_addr learned my addr 172.21.15.107:0/2909223955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16e531640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe16810f390 0x7fe1681a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 -- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 msgr2=0x7fe16819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 0x7fe16819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 -- 172.21.15.107:0/2909223955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe16810f390 msgr2=0x7fe1681a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe16810f390 0x7fe1681a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:51.880 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 -- 172.21.15.107:0/2909223955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a09e0 con 0x7fe168103890 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16dd30640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 0x7fe16819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16d52f640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16819ae80 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00d720 tx=0x7fe15c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c017070 con 0x7fe168103890 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a0cd0 con 0x7fe168103890 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7fe15c00de30 con 0x7fe168103890 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c005540 con 0x7fe168103890 2024-09-21T12:10:51.881 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.876+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a7390 con 0x7fe168103890 2024-09-21T12:10:51.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.877+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16806ac50 con 0x7fe168103890 2024-09-21T12:10:51.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.878+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7fe15c0099d0 con 0x7fe168103890 2024-09-21T12:10:51.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.879+0000 7fe166ffd640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fe15007b150 0x7fe15007d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:10:51.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.879+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(759..759 src has 1..759) ==== 9451+0+0 (secure 0 0 0) 0x7fe15c09eb90 con 0x7fe168103890 2024-09-21T12:10:51.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.879+0000 7fe16dd30640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fe15007b150 0x7fe15007d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:10:51.882 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.879+0000 7fe16dd30640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fe15007b150 0x7fe15007d610 secure :-1 s=READY pgs=3950 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e810 tx=0x7fe158009500 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:10:51.885 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:51.883+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe15c020020 con 0x7fe168103890 2024-09-21T12:10:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]': finished 2024-09-21T12:10:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:51 smithi193 ceph-mon[33333]: osdmap e758: 8 total, 8 up, 8 in 2024-09-21T12:10:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:51 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3585537981' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:52.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:52.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:51 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]': finished 2024-09-21T12:10:52.051 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:51 smithi193 ceph-mon[33333]: osdmap e759: 8 total, 8 up, 8 in 2024-09-21T12:10:52.063 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:52.062+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} v 0) -- 0x7fe1681a75f0 con 0x7fe168103890 2024-09-21T12:10:52.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]': finished 2024-09-21T12:10:52.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[41097]: osdmap e758: 8 total, 8 up, 8 in 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3585537981' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]': finished 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[41097]: osdmap e759: 8 total, 8 up, 8 in 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]': finished 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[30293]: osdmap e758: 8 total, 8 up, 8 in 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3585537981' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"} : dispatch 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "app": "rados"}]': finished 2024-09-21T12:10:52.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:51 smithi107 ceph-mon[30293]: osdmap e759: 8 total, 8 up, 8 in 2024-09-21T12:10:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:10:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:10:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:10:52.793 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:52.792+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 4a24a534-8178-47ec-9e91-28e33a5c3068 v760) ==== 223+0+0 (secure 0 0 0) 0x7fe15c069460 con 0x7fe168103890 2024-09-21T12:10:52.939 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:52.904+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} v 0) -- 0x7fe1681094f0 con 0x7fe168103890 2024-09-21T12:10:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:52 smithi193 ceph-mon[33333]: pgmap v1199: 188 pgs: 12 unknown, 176 active+clean; 746 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-21T12:10:53.050 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:52 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:10:53.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:52 smithi107 ceph-mon[30293]: pgmap v1199: 188 pgs: 12 unknown, 176 active+clean; 746 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-21T12:10:53.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:52 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:10:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:52 smithi107 ceph-mon[41097]: pgmap v1199: 188 pgs: 12 unknown, 176 active+clean; 746 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-09-21T12:10:53.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:52 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:10:53.796 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.794+0000 7fe166ffd640 1 -- 172.21.15.107:0/2909223955 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 4a24a534-8178-47ec-9e91-28e33a5c3068 v761) ==== 223+0+0 (secure 0 0 0) 0x7fe15c06e310 con 0x7fe168103890 2024-09-21T12:10:53.796 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 10 for pool 4a24a534-8178-47ec-9e91-28e33a5c3068 2024-09-21T12:10:53.799 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.797+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fe15007b150 msgr2=0x7fe15007d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:53.799 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.797+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fe15007b150 0x7fe15007d610 secure :-1 s=READY pgs=3950 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e810 tx=0x7fe158009500 comp rx=0 tx=0).stop 2024-09-21T12:10:53.799 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.797+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 msgr2=0x7fe16819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:10:53.799 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.797+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16819ae80 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00d720 tx=0x7fe15c00dbe0 comp rx=0 tx=0).stop 2024-09-21T12:10:53.800 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.798+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 shutdown_connections 2024-09-21T12:10:53.800 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7fe15007b150 0x7fe15007d610 unknown :-1 s=CLOSED pgs=3950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:53.800 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe16810f390 0x7fe1681a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:53.801 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe168103890 0x7fe16819ae80 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:53.801 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 --2- 172.21.15.107:0/2909223955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe16806a810 0x7fe16819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:10:53.801 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 >> 172.21.15.107:0/2909223955 conn(0x7fe1680776a0 msgr2=0x7fe168100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:10:53.801 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 shutdown_connections 2024-09-21T12:10:53.801 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:10:53.799+0000 7fe16ff94640 1 -- 172.21.15.107:0/2909223955 wait complete. 2024-09-21T12:10:53.813 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-21T12:10:54.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:10:54.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[41097]: osdmap e760: 8 total, 8 up, 8 in 2024-09-21T12:10:54.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[30293]: osdmap e760: 8 total, 8 up, 8 in 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:10:54.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:53 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:10:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:53 smithi193 ceph-mon[33333]: osdmap e760: 8 total, 8 up, 8 in 2024-09-21T12:10:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:53 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"} : dispatch 2024-09-21T12:10:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:53 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:10:54.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:53 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:10:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:54 smithi107 ceph-mon[30293]: pgmap v1202: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-21T12:10:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:54 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:10:55.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:54 smithi107 ceph-mon[30293]: osdmap e761: 8 total, 8 up, 8 in 2024-09-21T12:10:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:54 smithi107 ceph-mon[41097]: pgmap v1202: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-21T12:10:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:54 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:10:55.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:54 smithi107 ceph-mon[41097]: osdmap e761: 8 total, 8 up, 8 in 2024-09-21T12:10:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:54 smithi193 ceph-mon[33333]: pgmap v1202: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-21T12:10:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:54 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2909223955' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "field": "max_objects", "val": "10"}]': finished 2024-09-21T12:10:55.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:54 smithi193 ceph-mon[33333]: osdmap e761: 8 total, 8 up, 8 in 2024-09-21T12:10:57.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:56 smithi107 ceph-mon[30293]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-21T12:10:57.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:56 smithi107 ceph-mon[41097]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-21T12:10:57.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:56 smithi193 ceph-mon[33333]: pgmap v1204: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-21T12:10:59.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:10:58 smithi107 ceph-mon[30293]: pgmap v1205: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-21T12:10:59.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:10:58 smithi107 ceph-mon[41097]: pgmap v1205: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-21T12:10:59.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:10:58 smithi193 ceph-mon[33333]: pgmap v1205: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-21T12:10:59.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:10:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:11:01.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:00 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:01.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:00 smithi107 ceph-mon[30293]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-21T12:11:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:00 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:01.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:00 smithi107 ceph-mon[41097]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-21T12:11:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:00 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:01.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:00 smithi193 ceph-mon[33333]: pgmap v1206: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-21T12:11:02.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:11:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:11:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:11:03.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:02 smithi193 ceph-mon[33333]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-21T12:11:03.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:02 smithi107 ceph-mon[41097]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-21T12:11:03.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:02 smithi107 ceph-mon[30293]: pgmap v1207: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-21T12:11:05.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:04 smithi107 ceph-mon[30293]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:05.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:04 smithi107 ceph-mon[41097]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:05.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:04 smithi193 ceph-mon[33333]: pgmap v1208: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:07.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:06 smithi107 ceph-mon[41097]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 867 B/s rd, 0 op/s 2024-09-21T12:11:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:06 smithi107 ceph-mon[30293]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 867 B/s rd, 0 op/s 2024-09-21T12:11:07.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:06 smithi193 ceph-mon[33333]: pgmap v1209: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 867 B/s rd, 0 op/s 2024-09-21T12:11:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:08 smithi107 ceph-mon[41097]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:09.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:08 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:08 smithi107 ceph-mon[30293]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:09.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:08 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:08 smithi193 ceph-mon[33333]: pgmap v1210: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:09.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:08 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:09.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:11:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:11:11.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:10 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:11.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:10 smithi107 ceph-mon[41097]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:10 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:11.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:10 smithi107 ceph-mon[30293]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:10 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:11.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:10 smithi193 ceph-mon[33333]: pgmap v1211: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:11:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:11:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:11:13.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:12 smithi107 ceph-mon[41097]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:13.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:12 smithi107 ceph-mon[30293]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:13.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:12 smithi193 ceph-mon[33333]: pgmap v1212: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:15.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:14 smithi107 ceph-mon[41097]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:15.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:14 smithi107 ceph-mon[30293]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:15.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:14 smithi193 ceph-mon[33333]: pgmap v1213: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:17.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:16 smithi107 ceph-mon[41097]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:17.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:16 smithi107 ceph-mon[30293]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:17.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:16 smithi193 ceph-mon[33333]: pgmap v1214: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:19.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:18 smithi107 ceph-mon[41097]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:19.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:18 smithi107 ceph-mon[30293]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:19.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:18 smithi193 ceph-mon[33333]: pgmap v1215: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:19.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:11:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:11:21.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:20 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:21.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:20 smithi107 ceph-mon[41097]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:20 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:21.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:20 smithi107 ceph-mon[30293]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:20 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:21.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:20 smithi193 ceph-mon[33333]: pgmap v1216: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:22.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:11:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:11:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:11:23.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:22 smithi107 ceph-mon[41097]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:23.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:22 smithi107 ceph-mon[30293]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:23.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:22 smithi193 ceph-mon[33333]: pgmap v1217: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:23.815 INFO:tasks.workunit.client.0.smithi107.stderr:++ seq 1 10 2024-09-21T12:11:23.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:23.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj1 /etc/passwd 2024-09-21T12:11:23.857 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:23.858 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj2 /etc/passwd 2024-09-21T12:11:23.897 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:23.897 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj3 /etc/passwd 2024-09-21T12:11:23.934 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:23.934 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj4 /etc/passwd 2024-09-21T12:11:23.970 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:23.971 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj5 /etc/passwd 2024-09-21T12:11:24.006 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:24.006 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj6 /etc/passwd 2024-09-21T12:11:24.042 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:24.042 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj7 /etc/passwd 2024-09-21T12:11:24.077 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:24.078 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj8 /etc/passwd 2024-09-21T12:11:24.113 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:24.113 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj9 /etc/passwd 2024-09-21T12:11:24.149 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-21T12:11:24.149 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 4a24a534-8178-47ec-9e91-28e33a5c3068 put obj10 /etc/passwd 2024-09-21T12:11:24.184 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-21T12:11:24.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:23 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:24.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:23 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:24.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:23 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:25.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:24 smithi107 ceph-mon[30293]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:25.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:24 smithi107 ceph-mon[41097]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:25.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:24 smithi193 ceph-mon[33333]: pgmap v1218: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:27.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:26 smithi107 ceph-mon[30293]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:27.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:26 smithi107 ceph-mon[41097]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:27.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:26 smithi193 ceph-mon[33333]: pgmap v1219: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:29.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:28 smithi107 ceph-mon[30293]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:29.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:28 smithi107 ceph-mon[41097]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:29.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:28 smithi193 ceph-mon[33333]: pgmap v1220: 188 pgs: 188 active+clean; 759 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:29.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:11:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:11:31.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:30 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:31.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:30 smithi107 ceph-mon[41097]: pgmap v1221: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-21T12:11:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:30 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:31.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:30 smithi107 ceph-mon[30293]: pgmap v1221: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-21T12:11:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:30 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:31.301 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:30 smithi193 ceph-mon[33333]: pgmap v1221: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-21T12:11:32.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:11:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:11:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[41097]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-21T12:11:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[41097]: pool '4a24a534-8178-47ec-9e91-28e33a5c3068' is full (reached quota's max_objects: 10) 2024-09-21T12:11:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[41097]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:11:33.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[41097]: osdmap e762: 8 total, 8 up, 8 in 2024-09-21T12:11:33.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:11:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[30293]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-21T12:11:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[30293]: pool '4a24a534-8178-47ec-9e91-28e33a5c3068' is full (reached quota's max_objects: 10) 2024-09-21T12:11:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[30293]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:11:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[30293]: osdmap e762: 8 total, 8 up, 8 in 2024-09-21T12:11:33.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:32 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:11:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:32 smithi193 ceph-mon[33333]: pgmap v1222: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-21T12:11:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:32 smithi193 ceph-mon[33333]: pool '4a24a534-8178-47ec-9e91-28e33a5c3068' is full (reached quota's max_objects: 10) 2024-09-21T12:11:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:32 smithi193 ceph-mon[33333]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-21T12:11:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:32 smithi193 ceph-mon[33333]: osdmap e762: 8 total, 8 up, 8 in 2024-09-21T12:11:33.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:32 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:11:34.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:33 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:11:34.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:33 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:11:34.254 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:33 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:11:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:33 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:11:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:33 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:11:34.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:33 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:11:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:33 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:11:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:33 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:11:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:33 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:11:35.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:35 smithi193 ceph-mon[33333]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:35.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:35 smithi107 ceph-mon[30293]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:35.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:35 smithi107 ceph-mon[41097]: pgmap v1224: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:37.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:37 smithi193 ceph-mon[33333]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:37.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:37 smithi107 ceph-mon[30293]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:37.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:37 smithi107 ceph-mon[41097]: pgmap v1225: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:39 smithi193 ceph-mon[33333]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:39 smithi193 ceph-mon[33333]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:11:39.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:39 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:39.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:39 smithi107 ceph-mon[30293]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:39.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:39 smithi107 ceph-mon[30293]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:11:39.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:39 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:39.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:39 smithi107 ceph-mon[41097]: pgmap v1226: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-21T12:11:39.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:39 smithi107 ceph-mon[41097]: from='mgr.24491 ' entity='mgr.y' 2024-09-21T12:11:39.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:39 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:39.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:11:39 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:11:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:41 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:41.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:41 smithi193 ceph-mon[33333]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:41.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:41 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:41.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:41 smithi107 ceph-mon[41097]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:41.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:41 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:41.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:41 smithi107 ceph-mon[30293]: pgmap v1227: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:42.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:11:42 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:11:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:11:43.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:43 smithi193 ceph-mon[33333]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:43.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:43 smithi107 ceph-mon[30293]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:43.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:43 smithi107 ceph-mon[41097]: pgmap v1228: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:45.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:45 smithi193 ceph-mon[33333]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:11:45.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:45 smithi107 ceph-mon[41097]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:11:45.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:45 smithi107 ceph-mon[30293]: pgmap v1229: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T12:11:47.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:47 smithi193 ceph-mon[33333]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:47.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:47 smithi107 ceph-mon[41097]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:47.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:47 smithi107 ceph-mon[30293]: pgmap v1230: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:49.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:49 smithi193 ceph-mon[33333]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:49.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:49 smithi107 ceph-mon[30293]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:49.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:49 smithi107 ceph-mon[41097]: pgmap v1231: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:49.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:11:49 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:11:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:51 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:51.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:51 smithi193 ceph-mon[33333]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:51.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:51 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:51.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:51 smithi107 ceph-mon[30293]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:51 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:11:51.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:51 smithi107 ceph-mon[41097]: pgmap v1232: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:52.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:11:52 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:11:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:11:53.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:53 smithi107 ceph-mon[30293]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:53.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:53 smithi107 ceph-mon[41097]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:53.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:53 smithi193 ceph-mon[33333]: pgmap v1233: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:54.186 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 put threemore /etc/passwd 2024-09-21T12:11:54.229 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_bytes 0 2024-09-21T12:11:54.314 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- 172.21.15.107:0/1147104108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c104d50 msgr2=0x7f9e6c10e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:54.314 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- 172.21.15.107:0/1147104108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c104d50 0x7f9e6c10e750 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009be0 tx=0x7f9e6402a6c0 comp rx=0 tx=0).stop 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- 172.21.15.107:0/1147104108 shutdown_connections 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- 172.21.15.107:0/1147104108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c104d50 0x7f9e6c10e750 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- 172.21.15.107:0/1147104108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c1043b0 0x7f9e6c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- 172.21.15.107:0/1147104108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c101f90 0x7f9e6c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- 172.21.15.107:0/1147104108 >> 172.21.15.107:0/1147104108 conn(0x7f9e6c0776a0 msgr2=0x7f9e6c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- 172.21.15.107:0/1147104108 shutdown_connections 2024-09-21T12:11:54.315 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- 172.21.15.107:0/1147104108 wait complete. 2024-09-21T12:11:54.317 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 Processor -- start 2024-09-21T12:11:54.317 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- start start 2024-09-21T12:11:54.317 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 0x7f9e6c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c1043b0 0x7f9e6c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c104d50 0x7f9e6c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c110010 con 0x7f9e6c101f90 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c10fe90 con 0x7f9e6c1043b0 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.312+0000 7f9e7310e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c110190 con 0x7f9e6c104d50 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 0x7f9e6c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e716ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c104d50 0x7f9e6c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 0x7f9e6c1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60116/0 (socket says 172.21.15.107:60116) 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 -- 172.21.15.107:0/2930554725 learned_addr learned my addr 172.21.15.107:0/2930554725 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e6bfff640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c1043b0 0x7f9e6c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 -- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c104d50 msgr2=0x7f9e6c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:54.318 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c104d50 0x7f9e6c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 -- 172.21.15.107:0/2930554725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c1043b0 msgr2=0x7f9e6c1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c1043b0 0x7f9e6c1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c19c550 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e6bfff640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c1043b0 0x7f9e6c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e716ab640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c104d50 0x7f9e6c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e70eaa640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 0x7f9e6c1964b0 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c890 tx=0x7f9e6000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60004220 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9e60007e00 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c19c840 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e600047a0 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.313+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c1a30e0 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.315+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c102f40 con 0x7f9e6c101f90 2024-09-21T12:11:54.319 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.315+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f9e600079b0 con 0x7f9e6c101f90 2024-09-21T12:11:54.320 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.315+0000 7f9e69ffb640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e5c07d360 0x7f9e5c07f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:54.320 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.316+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(762..762 src has 1..762) ==== 9451+0+0 (secure 0 0 0) 0x7f9e6009f030 con 0x7f9e6c101f90 2024-09-21T12:11:54.320 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.316+0000 7f9e6bfff640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e5c07d360 0x7f9e5c07f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:54.320 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.316+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=763}) -- 0x7f9e5c089680 con 0x7f9e6c101f90 2024-09-21T12:11:54.320 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.316+0000 7f9e6bfff640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e5c07d360 0x7f9e5c07f820 secure :-1 s=READY pgs=3962 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c106ca0 tx=0x7f9e58009260 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:11:54.322 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.320+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e60014030 con 0x7f9e6c101f90 2024-09-21T12:11:54.502 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:54.499+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} v 0) -- 0x7f9e6c109d30 con 0x7f9e6c101f90 2024-09-21T12:11:54.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:54 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:54.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:54 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:54.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:54 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:11:55.060 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:55.057+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v763) ==== 217+0+0 (secure 0 0 0) 0x7f9e60069980 con 0x7f9e6c101f90 2024-09-21T12:11:55.060 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:55.058+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 8 ==== osd_map(763..763 src has 1..763) ==== 628+0+0 (secure 0 0 0) 0x7f9e600d6710 con 0x7f9e6c101f90 2024-09-21T12:11:55.060 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:55.058+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=764}) -- 0x7f9e5c08a5e0 con 0x7f9e6c101f90 2024-09-21T12:11:55.173 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:55.170+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} v 0) -- 0x7f9e6c1975d0 con 0x7f9e6c101f90 2024-09-21T12:11:55.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:55 smithi107 ceph-mon[30293]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:55.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:55 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:11:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:55 smithi107 ceph-mon[41097]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:55.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:55 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:11:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:55 smithi193 ceph-mon[33333]: pgmap v1234: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:11:55.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:55 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:11:56.075 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.072+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v764) ==== 217+0+0 (secure 0 0 0) 0x7f9e6006e830 con 0x7f9e6c101f90 2024-09-21T12:11:56.076 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_bytes = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:11:56.077 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.074+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 <== mon.0 v2:172.21.15.107:3300/0 10 ==== osd_map(764..764 src has 1..764) ==== 628+0+0 (secure 0 0 0) 0x7f9e6009c750 con 0x7f9e6c101f90 2024-09-21T12:11:56.077 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.074+0000 7f9e69ffb640 1 -- 172.21.15.107:0/2930554725 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=765}) -- 0x7f9e5c08a830 con 0x7f9e6c101f90 2024-09-21T12:11:56.080 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e5c07d360 msgr2=0x7f9e5c07f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:56.080 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e5c07d360 0x7f9e5c07f820 secure :-1 s=READY pgs=3962 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c106ca0 tx=0x7f9e58009260 comp rx=0 tx=0).stop 2024-09-21T12:11:56.080 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 msgr2=0x7f9e6c1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:56.080 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 0x7f9e6c1964b0 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c890 tx=0x7f9e6000cd50 comp rx=0 tx=0).stop 2024-09-21T12:11:56.080 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 shutdown_connections 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f9e5c07d360 0x7f9e5c07f820 unknown :-1 s=CLOSED pgs=3962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e6c104d50 0x7f9e6c19bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6c1043b0 0x7f9e6c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 --2- 172.21.15.107:0/2930554725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e6c101f90 0x7f9e6c1964b0 unknown :-1 s=CLOSED pgs=3052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.077+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 >> 172.21.15.107:0/2930554725 conn(0x7f9e6c0776a0 msgr2=0x7f9e6c1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.078+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 shutdown_connections 2024-09-21T12:11:56.081 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.078+0000 7f9e7310e640 1 -- 172.21.15.107:0/2930554725 wait complete. 2024-09-21T12:11:56.093 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 max_objects 0 2024-09-21T12:11:56.183 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 -- 172.21.15.107:0/2563040040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0109080 msgr2=0x7f7cb010f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:56.183 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/2563040040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0109080 0x7f7cb010f880 secure :-1 s=READY pgs=3053 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8009be0 tx=0x7f7ca802a6c0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 -- 172.21.15.107:0/2563040040 shutdown_connections 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/2563040040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0109080 0x7f7cb010f880 unknown :-1 s=CLOSED pgs=3053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/2563040040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0108700 0x7f7cb0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/2563040040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0102d00 0x7f7cb01030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 -- 172.21.15.107:0/2563040040 >> 172.21.15.107:0/2563040040 conn(0x7f7cb00fe740 msgr2=0x7f7cb0100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 -- 172.21.15.107:0/2563040040 shutdown_connections 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.178+0000 7f7cb7d24640 1 -- 172.21.15.107:0/2563040040 wait complete. 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 Processor -- start 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 -- start start 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0102d00 0x7f7cb00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:56.184 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 0x7f7cb00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 0x7f7cb0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0112210 con 0x7f7cb0108700 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0112090 con 0x7f7cb0102d00 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb7d24640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb0112390 con 0x7f7cb0109080 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 0x7f7cb0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb52bf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 0x7f7cb00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 0x7f7cb0075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:39380/0 (socket says 172.21.15.107:39380) 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 -- 172.21.15.107:0/1432194703 learned_addr learned my addr 172.21.15.107:0/1432194703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb52bf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 0x7f7cb00759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60134/0 (socket says 172.21.15.107:60134) 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb5ac0640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0102d00 0x7f7cb00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:56.185 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 -- 172.21.15.107:0/1432194703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0102d00 msgr2=0x7f7cb00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0102d00 0x7f7cb00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 -- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 msgr2=0x7f7cb00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 0x7f7cb00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb62c1640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb01a6b90 con 0x7f7cb0109080 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb5ac0640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0102d00 0x7f7cb00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.179+0000 7f7cb52bf640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 0x7f7cb00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.180+0000 7f7cb62c1640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 0x7f7cb0075f20 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004af0 tx=0x7f7ca8030ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.180+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8041020 con 0x7f7cb0109080 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.180+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb01a6e20 con 0x7f7cb0109080 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.180+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f7ca8003970 con 0x7f7cb0109080 2024-09-21T12:11:56.186 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.180+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8036040 con 0x7f7cb0109080 2024-09-21T12:11:56.187 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.180+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb01a7310 con 0x7f7cb0109080 2024-09-21T12:11:56.187 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.181+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb01030e0 con 0x7f7cb0109080 2024-09-21T12:11:56.188 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.182+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f7ca8044020 con 0x7f7cb0109080 2024-09-21T12:11:56.188 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.182+0000 7f7ca6ffd640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c9c07b150 0x7f7c9c07d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:11:56.188 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.182+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(764..764 src has 1..764) ==== 9451+0+0 (secure 0 0 0) 0x7f7ca80bfc60 con 0x7f7cb0109080 2024-09-21T12:11:56.188 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.182+0000 7f7cb5ac0640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c9c07b150 0x7f7c9c07d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:11:56.188 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.182+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=765}) -- 0x7f7c9c0873f0 con 0x7f7cb0109080 2024-09-21T12:11:56.188 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.182+0000 7f7cb5ac0640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c9c07b150 0x7f7c9c07d610 secure :-1 s=READY pgs=3963 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005fd0 tx=0x7f7cac009260 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:11:56.189 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.186+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ca804b020 con 0x7f7cb0109080 2024-09-21T12:11:56.368 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:56.365+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} v 0) -- 0x7f7cb01a7630 con 0x7f7cb0109080 2024-09-21T12:11:56.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:11:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:56 smithi107 ceph-mon[30293]: osdmap e763: 8 total, 8 up, 8 in 2024-09-21T12:11:56.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:56 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:11:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:11:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:56 smithi107 ceph-mon[41097]: osdmap e763: 8 total, 8 up, 8 in 2024-09-21T12:11:56.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:56 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:11:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:11:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:56 smithi193 ceph-mon[33333]: osdmap e763: 8 total, 8 up, 8 in 2024-09-21T12:11:56.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:56 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"} : dispatch 2024-09-21T12:11:57.078 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:57.075+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 7 ==== osd_map(765..765 src has 1..765) ==== 628+0+0 (secure 0 0 0) 0x7f7ca8003d90 con 0x7f7cb0109080 2024-09-21T12:11:57.078 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:57.075+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=766}) -- 0x7f7c9c087fa0 con 0x7f7cb0109080 2024-09-21T12:11:57.084 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:57.081+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v765) ==== 221+0+0 (secure 0 0 0) 0x7f7ca808a530 con 0x7f7cb0109080 2024-09-21T12:11:57.196 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:57.193+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} v 0) -- 0x7f7cb01098f0 con 0x7f7cb0109080 2024-09-21T12:11:57.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[30293]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:57.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:11:57.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[30293]: osdmap e764: 8 total, 8 up, 8 in 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1432194703' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[41097]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[41097]: osdmap e764: 8 total, 8 up, 8 in 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1432194703' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:57.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:57 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:57 smithi193 ceph-mon[33333]: pgmap v1236: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:11:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/2930554725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_bytes", "val": "0"}]': finished 2024-09-21T12:11:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:57 smithi193 ceph-mon[33333]: osdmap e764: 8 total, 8 up, 8 in 2024-09-21T12:11:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:57 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1432194703' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:57.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:57 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:58.094 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.091+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 9 ==== osd_map(766..766 src has 1..766) ==== 628+0+0 (secure 0 0 0) 0x7f7ca80bd380 con 0x7f7cb0109080 2024-09-21T12:11:58.094 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.091+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=767}) -- 0x7f7c9c0883d0 con 0x7f7cb0109080 2024-09-21T12:11:58.101 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.098+0000 7f7ca6ffd640 1 -- 172.21.15.107:0/1432194703 <== mon.2 v2:172.21.15.107:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 v766) ==== 221+0+0 (secure 0 0 0) 0x7f7ca808f3e0 con 0x7f7cb0109080 2024-09-21T12:11:58.101 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 0 for pool abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 2024-09-21T12:11:58.104 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c9c07b150 msgr2=0x7f7c9c07d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c9c07b150 0x7f7c9c07d610 secure :-1 s=READY pgs=3963 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005fd0 tx=0x7f7cac009260 comp rx=0 tx=0).stop 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 msgr2=0x7f7cb0075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 0x7f7cb0075f20 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8004af0 tx=0x7f7ca8030ad0 comp rx=0 tx=0).stop 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 shutdown_connections 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f7c9c07b150 0x7f7c9c07d610 unknown :-1 s=CLOSED pgs=3963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb0109080 0x7f7cb0075f20 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb0108700 0x7f7cb00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 --2- 172.21.15.107:0/1432194703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cb0102d00 0x7f7cb00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:11:58.105 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 >> 172.21.15.107:0/1432194703 conn(0x7f7cb00fe740 msgr2=0x7f7cb0101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:11:58.106 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 shutdown_connections 2024-09-21T12:11:58.106 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:11:58.101+0000 7f7cb7d24640 1 -- 172.21.15.107:0/1432194703 wait complete. 2024-09-21T12:11:58.114 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-21T12:11:58.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[30293]: osdmap e765: 8 total, 8 up, 8 in 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1432194703' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[41097]: osdmap e765: 8 total, 8 up, 8 in 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1432194703' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:58.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:58 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:11:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:58 smithi193 ceph-mon[33333]: osdmap e765: 8 total, 8 up, 8 in 2024-09-21T12:11:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:58 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1432194703' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:58.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:58 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"} : dispatch 2024-09-21T12:11:59.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:59 smithi193 ceph-mon[33333]: pgmap v1239: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:59.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:59 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:11:59.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:11:59 smithi193 ceph-mon[33333]: osdmap e766: 8 total, 8 up, 8 in 2024-09-21T12:11:59.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:59 smithi107 ceph-mon[41097]: pgmap v1239: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:59.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:59 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:11:59.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:11:59 smithi107 ceph-mon[41097]: osdmap e766: 8 total, 8 up, 8 in 2024-09-21T12:11:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:59 smithi107 ceph-mon[30293]: pgmap v1239: 188 pgs: 188 active+clean; 884 KiB data, 825 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:11:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:59 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "field": "max_objects", "val": "0"}]': finished 2024-09-21T12:11:59.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:11:59 smithi107 ceph-mon[30293]: osdmap e766: 8 total, 8 up, 8 in 2024-09-21T12:11:59.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:11:59 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:12:01.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:01 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:01.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:01 smithi107 ceph-mon[41097]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-21T12:12:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:01 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:01.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:01 smithi107 ceph-mon[30293]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-21T12:12:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:01 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:01.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:01 smithi193 ceph-mon[33333]: pgmap v1241: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-21T12:12:02.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:02 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:12:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:12:03.352 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:03 smithi193 ceph-mon[33333]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-21T12:12:03.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:03 smithi107 ceph-mon[30293]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-21T12:12:03.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:03 smithi107 ceph-mon[41097]: pgmap v1242: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-21T12:12:05.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:05 smithi107 ceph-mon[41097]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-21T12:12:05.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:05 smithi107 ceph-mon[30293]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-21T12:12:05.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:05 smithi193 ceph-mon[33333]: pgmap v1243: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-21T12:12:07.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:07 smithi107 ceph-mon[30293]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-21T12:12:07.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:07 smithi107 ceph-mon[41097]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-21T12:12:07.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:07 smithi193 ceph-mon[33333]: pgmap v1244: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-21T12:12:09.453 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:09 smithi193 ceph-mon[33333]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-21T12:12:09.453 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:09 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:12:09.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:09 smithi107 ceph-mon[41097]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-21T12:12:09.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:09 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:12:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:09 smithi107 ceph-mon[30293]: pgmap v1245: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-21T12:12:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:09 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:12:09.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:12:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:12:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:11 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:11.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:11 smithi107 ceph-mon[41097]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 1 op/s 2024-09-21T12:12:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:11 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:11.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:11 smithi107 ceph-mon[30293]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 1 op/s 2024-09-21T12:12:11.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:11 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:11.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:11 smithi193 ceph-mon[33333]: pgmap v1246: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 1 op/s 2024-09-21T12:12:12.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:12 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:12:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:12:13.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:13 smithi107 ceph-mon[30293]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:12:13.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:13 smithi107 ceph-mon[41097]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:12:13.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:13 smithi193 ceph-mon[33333]: pgmap v1247: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-21T12:12:15.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:15 smithi107 ceph-mon[30293]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:15.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:15 smithi107 ceph-mon[41097]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:15.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:15 smithi193 ceph-mon[33333]: pgmap v1248: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:17.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:17 smithi107 ceph-mon[41097]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:17.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:17 smithi107 ceph-mon[30293]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:17.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:17 smithi193 ceph-mon[33333]: pgmap v1249: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:19.456 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:19 smithi193 ceph-mon[33333]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:19.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:19 smithi107 ceph-mon[30293]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:19.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:19 smithi107 ceph-mon[41097]: pgmap v1250: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:19.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:12:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:12:21.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:21 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:21.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:21 smithi107 ceph-mon[30293]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:21.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:21 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:21.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:21 smithi107 ceph-mon[41097]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:21.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:21 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:21.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:21 smithi193 ceph-mon[33333]: pgmap v1251: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:22.753 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:22 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:12:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:12:23.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:23 smithi107 ceph-mon[30293]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:23.504 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:23 smithi107 ceph-mon[41097]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:23.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:23 smithi193 ceph-mon[33333]: pgmap v1252: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:24.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:24 smithi107 ceph-mon[30293]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:12:24.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:24 smithi107 ceph-mon[41097]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:12:24.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:24 smithi193 ceph-mon[33333]: from='mgr.24491 172.21.15.107:0/3985990014' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:12:25.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:25 smithi107 ceph-mon[30293]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:25.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:25 smithi107 ceph-mon[41097]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:25.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:25 smithi193 ceph-mon[33333]: pgmap v1253: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:12:27.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:27 smithi107 ceph-mon[30293]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:27.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:27 smithi107 ceph-mon[41097]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:27.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:27 smithi193 ceph-mon[33333]: pgmap v1254: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:28.116 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool delete abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 abb60f59-7773-47b1-b6bb-dfb9b5cd1a87 --yes-i-really-really-mean-it 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.207+0000 7f911f194640 1 -- 172.21.15.107:0/4169749809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 msgr2=0x7f911810f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.207+0000 7f911f194640 1 --2- 172.21.15.107:0/4169749809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f911810f880 secure :-1 s=READY pgs=3088 cs=0 l=1 rev1=1 crypto rx=0x7f9110009ca0 tx=0x7f911002a8c0 comp rx=0 tx=0).stop 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- 172.21.15.107:0/4169749809 shutdown_connections 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 --2- 172.21.15.107:0/4169749809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f911810f880 unknown :-1 s=CLOSED pgs=3088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 --2- 172.21.15.107:0/4169749809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f9118108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 --2- 172.21.15.107:0/4169749809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9118102d00 0x7f91181030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- 172.21.15.107:0/4169749809 >> 172.21.15.107:0/4169749809 conn(0x7f91180fe740 msgr2=0x7f9118100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- 172.21.15.107:0/4169749809 shutdown_connections 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- 172.21.15.107:0/4169749809 wait complete. 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 Processor -- start 2024-09-21T12:12:28.210 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- start start 2024-09-21T12:12:28.212 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9118102d00 0x7f91180754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f91180759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f9118075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9118112210 con 0x7f9118108700 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9118112090 con 0x7f9118102d00 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f911f194640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7f9118112390 con 0x7f9118109080 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.208+0000 7f9117fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f91180759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f911d731640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f9118075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f91180759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51372/0 (socket says 172.21.15.107:51372) 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 -- 172.21.15.107:0/1889145607 learned_addr learned my addr 172.21.15.107:0/1889145607 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f911cf30640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9118102d00 0x7f91180754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 -- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 msgr2=0x7f9118075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f9118075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 -- 172.21.15.107:0/1889145607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9118102d00 msgr2=0x7f91180754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:28.213 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9118102d00 0x7f91180754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91181a6b60 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f911d731640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f9118075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9117fff640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f91180759e0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f910800ea60 tx=0x7f910800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910800cde0 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7f9108004470 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91181a6e50 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108010430 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.209+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91181a7310 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.210+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91181030e0 con 0x7f9118108700 2024-09-21T12:12:28.214 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.211+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7f9108003f60 con 0x7f9118108700 2024-09-21T12:12:28.215 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.211+0000 7f9115ffb640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f910407b150 0x7f910407d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:28.215 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.211+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(766..766 src has 1..766) ==== 9451+0+0 (secure 0 0 0) 0x7f910809f300 con 0x7f9118108700 2024-09-21T12:12:28.215 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.211+0000 7f911cf30640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f910407b150 0x7f910407d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:28.215 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.212+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=767}) -- 0x7f91040873f0 con 0x7f9118108700 2024-09-21T12:12:28.215 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.212+0000 7f911cf30640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f910407b150 0x7f910407d610 secure :-1 s=READY pgs=3964 cs=0 l=1 rev1=1 crypto rx=0x7f9100005fd0 tx=0x7f910000b390 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:12:28.217 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.216+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9108069bd0 con 0x7f9118108700 2024-09-21T12:12:28.397 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:28.395+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} v 0) -- 0x7f911802e5c0 con 0x7f9118108700 2024-09-21T12:12:29.146 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.145+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true}]=0 pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' removed v767) ==== 248+0+0 (secure 0 0 0) 0x7f910806ea80 con 0x7f9118108700 2024-09-21T12:12:29.149 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.147+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 8 ==== osd_map(767..767 src has 1..767) ==== 296+0+0 (secure 0 0 0) 0x7f91080d6710 con 0x7f9118108700 2024-09-21T12:12:29.149 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.147+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=768}) -- 0x7f91040880e0 con 0x7f9118108700 2024-09-21T12:12:29.263 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.261+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} v 0) -- 0x7f911802e840 con 0x7f9118108700 2024-09-21T12:12:29.263 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.262+0000 7f9115ffb640 1 -- 172.21.15.107:0/1889145607 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true}]=0 pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' does not exist v767) ==== 255+0+0 (secure 0 0 0) 0x7f910806c0c0 con 0x7f9118108700 2024-09-21T12:12:29.263 INFO:tasks.workunit.client.0.smithi107.stderr:pool 'abb60f59-7773-47b1-b6bb-dfb9b5cd1a87' does not exist 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f910407b150 msgr2=0x7f910407d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f910407b150 0x7f910407d610 secure :-1 s=READY pgs=3964 cs=0 l=1 rev1=1 crypto rx=0x7f9100005fd0 tx=0x7f910000b390 comp rx=0 tx=0).stop 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 msgr2=0x7f91180759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f91180759e0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7f910800ea60 tx=0x7f910800ef20 comp rx=0 tx=0).stop 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 shutdown_connections 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7f910407b150 0x7f910407d610 unknown :-1 s=CLOSED pgs=3964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9118109080 0x7f9118075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118108700 0x7f91180759e0 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 --2- 172.21.15.107:0/1889145607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9118102d00 0x7f91180754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.266 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 >> 172.21.15.107:0/1889145607 conn(0x7f91180fe740 msgr2=0x7f9118100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:12:29.267 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 shutdown_connections 2024-09-21T12:12:29.267 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.264+0000 7f911f194640 1 -- 172.21.15.107:0/1889145607 wait complete. 2024-09-21T12:12:29.274 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool delete 4a24a534-8178-47ec-9e91-28e33a5c3068 4a24a534-8178-47ec-9e91-28e33a5c3068 --yes-i-really-really-mean-it 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 -- 172.21.15.107:0/1710969296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 msgr2=0x7ff3e01030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/1710969296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e01030e0 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009be0 tx=0x7ff3d002a6c0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 -- 172.21.15.107:0/1710969296 shutdown_connections 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/1710969296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0109080 0x7ff3e010f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/1710969296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0108700 0x7ff3e0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/1710969296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e01030e0 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 -- 172.21.15.107:0/1710969296 >> 172.21.15.107:0/1710969296 conn(0x7ff3e00fe740 msgr2=0x7ff3e0100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:12:29.360 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 -- 172.21.15.107:0/1710969296 shutdown_connections 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.356+0000 7ff3e58cd640 1 -- 172.21.15.107:0/1710969296 wait complete. 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 Processor -- start 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 -- start start 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0108700 0x7ff3e00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0109080 0x7ff3e0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0112210 con 0x7ff3e0109080 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0112090 con 0x7ff3e0108700 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3e58cd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e0112390 con 0x7ff3e0102d00 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3deffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3deffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:57214/0 (socket says 172.21.15.107:57214) 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3deffd640 1 -- 172.21.15.107:0/3465686325 learned_addr learned my addr 172.21.15.107:0/3465686325 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3df7fe640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0109080 0x7ff3e0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:29.361 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.357+0000 7ff3de7fc640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0108700 0x7ff3e00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3deffd640 1 -- 172.21.15.107:0/3465686325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0108700 msgr2=0x7ff3e00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3deffd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0108700 0x7ff3e00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3deffd640 1 -- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0109080 msgr2=0x7ff3e0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3deffd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0109080 0x7ff3e0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3deffd640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e01a6b80 con 0x7ff3e0102d00 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3df7fe640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0109080 0x7ff3e0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3de7fc640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0108700 0x7ff3e00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3deffd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e00754a0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0004af0 tx=0x7ff3d0030ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0041020 con 0x7ff3e0102d00 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e01a6e10 con 0x7ff3e0102d00 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(23 keys) ==== 1003+0+0 (secure 0 0 0) 0x7ff3d0003970 con 0x7ff3e0102d00 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0036040 con 0x7ff3e0102d00 2024-09-21T12:12:29.362 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.358+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e01a7270 con 0x7ff3e0102d00 2024-09-21T12:12:29.363 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.359+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e01030e0 con 0x7ff3e0102d00 2024-09-21T12:12:29.363 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.360+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 23) ==== 103292+0+0 (secure 0 0 0) 0x7ff3d0044020 con 0x7ff3e0102d00 2024-09-21T12:12:29.363 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.360+0000 7ff3e48cb640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7ff3c807b150 0x7ff3c807d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T12:12:29.363 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.360+0000 7ff3de7fc640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7ff3c807b150 0x7ff3c807d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T12:12:29.364 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.360+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(767..767 src has 1..767) ==== 9063+0+0 (secure 0 0 0) 0x7ff3d00bfc50 con 0x7ff3e0102d00 2024-09-21T12:12:29.364 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.360+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=768}) -- 0x7ff3c8086bd0 con 0x7ff3e0102d00 2024-09-21T12:12:29.364 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.361+0000 7ff3de7fc640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7ff3c807b150 0x7ff3c807d610 secure :-1 s=READY pgs=3965 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0076f90 tx=0x7ff3d4009260 comp rx=0 tx=0).ready entity=mgr.24491 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T12:12:29.366 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.365+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff3d00c5020 con 0x7ff3e0102d00 2024-09-21T12:12:29.457 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:29 smithi193 ceph-mon[33333]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:29.457 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:29 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:29.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:29 smithi107 ceph-mon[30293]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:29.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:29 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:29.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:29 smithi107 ceph-mon[41097]: pgmap v1255: 188 pgs: 188 active+clean; 897 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:12:29.503 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:29 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:29.549 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:29.547+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff3e002e5c0 con 0x7ff3e0102d00 2024-09-21T12:12:29.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:12:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug there is no tcmu-runner data available 2024-09-21T12:12:30.176 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.174+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 7 ==== osd_map(768..768 src has 1..768) ==== 296+0+0 (secure 0 0 0) 0x7ff3d00bd4f0 con 0x7ff3e0102d00 2024-09-21T12:12:30.176 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.174+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=769}) -- 0x7ff3c8087400 con 0x7ff3e0102d00 2024-09-21T12:12:30.181 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.179+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true}]=0 pool '4a24a534-8178-47ec-9e91-28e33a5c3068' removed v768) ==== 248+0+0 (secure 0 0 0) 0x7ff3d004b020 con 0x7ff3e0102d00 2024-09-21T12:12:30.293 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.291+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff3e010ae60 con 0x7ff3e0102d00 2024-09-21T12:12:30.294 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.292+0000 7ff3e48cb640 1 -- 172.21.15.107:0/3465686325 <== mon.2 v2:172.21.15.107:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true}]=0 pool '4a24a534-8178-47ec-9e91-28e33a5c3068' does not exist v768) ==== 255+0+0 (secure 0 0 0) 0x7ff3d008a6a0 con 0x7ff3e0102d00 2024-09-21T12:12:30.294 INFO:tasks.workunit.client.0.smithi107.stderr:pool '4a24a534-8178-47ec-9e91-28e33a5c3068' does not exist 2024-09-21T12:12:30.296 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7ff3c807b150 msgr2=0x7ff3c807d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:30.296 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7ff3c807b150 0x7ff3c807d610 secure :-1 s=READY pgs=3965 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0076f90 tx=0x7ff3d4009260 comp rx=0 tx=0).stop 2024-09-21T12:12:30.296 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 msgr2=0x7ff3e00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T12:12:30.296 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e00754a0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0004af0 tx=0x7ff3d0030ad0 comp rx=0 tx=0).stop 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 shutdown_connections 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:6800/271304053,v1:172.21.15.107:6801/271304053] conn(0x7ff3c807b150 0x7ff3c807d610 unknown :-1 s=CLOSED pgs=3965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3e0109080 0x7ff3e0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e0108700 0x7ff3e00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 --2- 172.21.15.107:0/3465686325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3e0102d00 0x7ff3e00754a0 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.294+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 >> 172.21.15.107:0/3465686325 conn(0x7ff3e00fe740 msgr2=0x7ff3e0100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.295+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 shutdown_connections 2024-09-21T12:12:30.297 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-21T12:12:30.295+0000 7ff3e58cd640 1 -- 172.21.15.107:0/3465686325 wait complete. 2024-09-21T12:12:30.305 INFO:tasks.workunit.client.0.smithi107.stdout:OK 2024-09-21T12:12:30.306 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo OK 2024-09-21T12:12:30.306 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T12:12:30.307 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T12:12:30.373 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-21T12:12:30.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T12:12:30.433 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[41097]: osdmap e767: 8 total, 8 up, 8 in 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3465686325' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[30293]: osdmap e767: 8 total, 8 up, 8 in 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3465686325' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:30 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:12:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:30 smithi193 ceph-mon[33333]: osdmap e767: 8 total, 8 up, 8 in 2024-09-21T12:12:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/1889145607' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "pool2": "abb60f59-7773-47b1-b6bb-dfb9b5cd1a87", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:30 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3465686325' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:30 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:30.699 DEBUG:teuthology.parallel:result is None 2024-09-21T12:12:30.700 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T12:12:30.768 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T12:12:30.768 DEBUG:teuthology.orchestra.run.smithi107:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-21T12:12:30.827 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T12:12:30.828 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T12:12:30.842 INFO:tasks.cephadm:Teardown begin 2024-09-21T12:12:30.843 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:12:30.891 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:12:30.925 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T12:12:30.925 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T12:12:30.947 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T12:12:30.983 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T12:12:30.984 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T12:12:30.984 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a 2024-09-21T12:12:31.269 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:31.269 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: pgmap v1257: 176 pgs: 176 active+clean; 708 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:12:31.269 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:12:31.269 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: osdmap e768: 8 total, 8 up, 8 in 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: from='client.? 172.21.15.107:0/3465686325' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[41097]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 systemd[1]: Stopping Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: pgmap v1257: 176 pgs: 176 active+clean; 708 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:12:31.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:12:31.271 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: osdmap e768: 8 total, 8 up, 8 in 2024-09-21T12:12:31.271 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: from='client.? 172.21.15.107:0/3465686325' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:31.271 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-mon[30293]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:31.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:12:31.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: pgmap v1257: 176 pgs: 176 active+clean; 708 KiB data, 844 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T12:12:31.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-21T12:12:31.550 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:12:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: osdmap e768: 8 total, 8 up, 8 in 2024-09-21T12:12:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: from='client.? 172.21.15.107:0/3465686325' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:31.551 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:31 smithi193 ceph-mon[33333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool delete", "pool": "4a24a534-8178-47ec-9e91-28e33a5c3068", "pool2": "4a24a534-8178-47ec-9e91-28e33a5c3068", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:12:31.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a[30270]: 2024-09-21T12:12:31.266+0000 7f24a6e12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:31.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a[30270]: 2024-09-21T12:12:31.266+0000 7f24a6e12640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T12:12:31.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 podman[148563]: 2024-09-21 12:12:31.349296144 +0000 UTC m=+0.175983280 container stop 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T12:12:31.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 podman[148563]: 2024-09-21 12:12:31.374836866 +0000 UTC m=+0.201523990 container died 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T12:12:32.003 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 podman[148563]: 2024-09-21 12:12:31.840633925 +0000 UTC m=+0.667321047 container cleanup 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T12:12:32.004 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:31 smithi107 bash[148563]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a 2024-09-21T12:12:32.256 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service' 2024-09-21T12:12:32.277 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: ::ffff:172.21.15.193 - - [21/Sep/2024:12:12:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:12:32.277 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:32 smithi107 podman[148580]: 2024-09-21 12:12:32.025485177 +0000 UTC m=+0.667011542 container remove 0c273e847eecc723853344af9f63dccb01a6e1ef2c1c574bb60c5612be34149c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:12:32.277 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:32 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T12:12:32.278 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:32 smithi107 systemd[1]: Stopped Ceph mon.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T12:12:32.278 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 21 12:12:32 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.a.service: Consumed 37.213s CPU time. 2024-09-21T12:12:32.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:32.293 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T12:12:32.293 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-21T12:12:32.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.c 2024-09-21T12:12:32.614 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:32 smithi107 systemd[1]: Stopping Ceph mon.c for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:33.003 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c[41074]: 2024-09-21T12:12:32.612+0000 7fd5ca07a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c[41074]: 2024-09-21T12:12:32.612+0000 7fd5ca07a640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:32 smithi107 podman[148695]: 2024-09-21 12:12:32.807439804 +0000 UTC m=+0.296257291 container died 6a100a689a79a5dad20ebfb05921809a4a79cf8d7fef3cd297c3dcf9ed94a14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:12:12:32] ENGINE Bus STOPPING 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:12:12:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:12:12:32] ENGINE Bus STOPPED 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:12:12:32] ENGINE Bus STARTING 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:12:12:32] ENGINE Serving on http://:::9283 2024-09-21T12:12:33.004 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:32 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: [21/Sep/2024:12:12:32] ENGINE Bus STARTED 2024-09-21T12:12:33.455 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:33 smithi107 podman[148695]: 2024-09-21 12:12:33.169833854 +0000 UTC m=+0.658651334 container cleanup 6a100a689a79a5dad20ebfb05921809a4a79cf8d7fef3cd297c3dcf9ed94a14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:12:33.455 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 21 12:12:33 smithi107 bash[148695]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-c 2024-09-21T12:12:33.596 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.c.service' 2024-09-21T12:12:33.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:33.630 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-21T12:12:33.630 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-21T12:12:33.630 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.b 2024-09-21T12:12:33.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:33 smithi193 systemd[1]: Stopping Ceph mon.b for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:33 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-b[33298]: 2024-09-21T12:12:33.956+0000 7f0a9f59e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:33 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-b[33298]: 2024-09-21T12:12:33.956+0000 7f0a9f59e640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T12:12:34.300 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:34 smithi193 podman[67900]: 2024-09-21 12:12:34.079933586 +0000 UTC m=+0.250243464 container died 7a8743e02fa4eabaf4b34517eec6d1b4ee466ff6972e058ce8138eef9aa77479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:12:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:34 smithi193 podman[67900]: 2024-09-21 12:12:34.589155263 +0000 UTC m=+0.759465135 container cleanup 7a8743e02fa4eabaf4b34517eec6d1b4ee466ff6972e058ce8138eef9aa77479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T12:12:34.800 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 21 12:12:34 smithi193 bash[67900]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-mon-b 2024-09-21T12:12:35.021 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mon.b.service' 2024-09-21T12:12:35.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:35.065 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-21T12:12:35.065 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T12:12:35.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y 2024-09-21T12:12:35.402 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:35 smithi107 systemd[1]: Stopping Ceph mgr.y for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:35.718 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:35 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T12:12:35.400+0000 7f44da57c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:35.718 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:35 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y[30540]: 2024-09-21T12:12:35.400+0000 7f44da57c640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T12:12:35.718 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:35 smithi107 podman[148837]: 2024-09-21 12:12:35.43266441 +0000 UTC m=+0.178988754 container died e60ab524a9f6ad9aac3f0a69d75f152cf14152b50b5aca11829b16f86b2295cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T12:12:36.003 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:35 smithi107 podman[148837]: 2024-09-21 12:12:35.807911165 +0000 UTC m=+0.554235503 container cleanup e60ab524a9f6ad9aac3f0a69d75f152cf14152b50b5aca11829b16f86b2295cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T12:12:36.003 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 12:12:35 smithi107 bash[148837]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-y 2024-09-21T12:12:36.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.y.service' 2024-09-21T12:12:36.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:36.238 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T12:12:36.238 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T12:12:36.238 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.x 2024-09-21T12:12:36.550 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 12:12:36 smithi193 systemd[1]: Stopping Ceph mgr.x for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:36.888 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 12:12:36 smithi193 podman[68031]: 2024-09-21 12:12:36.619062419 +0000 UTC m=+0.188458625 container died 5edb09b6166ce23a84ed9a30796ca69f95b2f25f57022b1861413eca49dcb706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T12:12:37.280 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 12:12:36 smithi193 podman[68031]: 2024-09-21 12:12:36.977901197 +0000 UTC m=+0.547297392 container cleanup 5edb09b6166ce23a84ed9a30796ca69f95b2f25f57022b1861413eca49dcb706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T12:12:37.280 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 21 12:12:36 smithi193 bash[68031]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-mgr-x 2024-09-21T12:12:37.448 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@mgr.x.service' 2024-09-21T12:12:37.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:37.484 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T12:12:37.484 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T12:12:37.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.0 2024-09-21T12:12:37.790 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:37 smithi107 systemd[1]: Stopping Ceph osd.0 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:38.253 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:37 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0[44809]: 2024-09-21T12:12:37.788+0000 7efee7770640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:38.253 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:37 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0[44809]: 2024-09-21T12:12:37.788+0000 7efee7770640 -1 osd.0 768 *** Got signal Terminated *** 2024-09-21T12:12:38.254 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:37 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0[44809]: 2024-09-21T12:12:37.788+0000 7efee7770640 -1 osd.0 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:12:43.632 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:43 smithi107 podman[148969]: 2024-09-21 12:12:43.323323283 +0000 UTC m=+5.649310450 container died 8c28ef404617a3c4daab1c045db6983e8f9c395af3d586437e0836e453716841 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=) 2024-09-21T12:12:43.939 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:43 smithi107 podman[148969]: 2024-09-21 12:12:43.745396281 +0000 UTC m=+6.071383437 container cleanup 8c28ef404617a3c4daab1c045db6983e8f9c395af3d586437e0836e453716841 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T12:12:43.939 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:43 smithi107 bash[148969]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0 2024-09-21T12:12:44.253 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:43 smithi107 podman[148989]: 2024-09-21 12:12:43.938374296 +0000 UTC m=+0.611375969 container remove 8c28ef404617a3c4daab1c045db6983e8f9c395af3d586437e0836e453716841 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T12:12:44.615 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:44 smithi107 podman[149085]: 2024-09-21 12:12:44.262476301 +0000 UTC m=+0.015705525 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:12:44.615 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:44 smithi107 podman[149085]: 2024-09-21 12:12:44.364718763 +0000 UTC m=+0.117948020 container create 0e17789af72ad8c5b32674ce7bcc1e0826b15e4422b6fb00ae25ab77e5b27bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T12:12:44.616 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:44 smithi107 podman[149085]: 2024-09-21 12:12:44.609841194 +0000 UTC m=+0.363070419 container init 0e17789af72ad8c5b32674ce7bcc1e0826b15e4422b6fb00ae25ab77e5b27bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T12:12:44.616 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:44 smithi107 podman[149085]: 2024-09-21 12:12:44.614451172 +0000 UTC m=+0.367680387 container start 0e17789af72ad8c5b32674ce7bcc1e0826b15e4422b6fb00ae25ab77e5b27bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T12:12:45.003 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:44 smithi107 podman[149085]: 2024-09-21 12:12:44.709001482 +0000 UTC m=+0.462230698 container attach 0e17789af72ad8c5b32674ce7bcc1e0826b15e4422b6fb00ae25ab77e5b27bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:12:45.003 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:44 smithi107 podman[149085]: 2024-09-21 12:12:44.810463916 +0000 UTC m=+0.563693145 container died 0e17789af72ad8c5b32674ce7bcc1e0826b15e4422b6fb00ae25ab77e5b27bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:12:45.354 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 21 12:12:45 smithi107 podman[149129]: 2024-09-21 12:12:45.337067564 +0000 UTC m=+0.521078320 container remove 0e17789af72ad8c5b32674ce7bcc1e0826b15e4422b6fb00ae25ab77e5b27bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T12:12:45.359 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.0.service' 2024-09-21T12:12:45.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:45.397 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T12:12:45.397 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T12:12:45.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.1 2024-09-21T12:12:45.628 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:45 smithi107 systemd[1]: Stopping Ceph osd.1 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:46.004 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:45 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1[50615]: 2024-09-21T12:12:45.758+0000 7f1093aee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:46.004 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:45 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1[50615]: 2024-09-21T12:12:45.758+0000 7f1093aee640 -1 osd.1 768 *** Got signal Terminated *** 2024-09-21T12:12:46.004 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:45 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1[50615]: 2024-09-21T12:12:45.758+0000 7f1093aee640 -1 osd.1 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:12:51.601 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:51 smithi107 podman[149216]: 2024-09-21 12:12:51.293508672 +0000 UTC m=+5.653914884 container died 3e6b3eec0d324e798febcfca43576ca579bcb846dd043bb749c037fbde49e37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T12:12:51.925 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:51 smithi107 podman[149216]: 2024-09-21 12:12:51.689702195 +0000 UTC m=+6.050108406 container cleanup 3e6b3eec0d324e798febcfca43576ca579bcb846dd043bb749c037fbde49e37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:12:51.925 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:51 smithi107 bash[149216]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1 2024-09-21T12:12:52.251 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:51 smithi107 podman[149248]: 2024-09-21 12:12:51.924318123 +0000 UTC m=+0.632469681 container remove 3e6b3eec0d324e798febcfca43576ca579bcb846dd043bb749c037fbde49e37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T12:12:52.503 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:52 smithi107 podman[149342]: 2024-09-21 12:12:52.153271348 +0000 UTC m=+0.015591375 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:12:52.504 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:52 smithi107 podman[149342]: 2024-09-21 12:12:52.266674851 +0000 UTC m=+0.128994866 container create 5c46e23442ad143a3478403b4f0198f9dbb92cbba610a4bbd5384bf49cb58f7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T12:12:52.989 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:52 smithi107 podman[149342]: 2024-09-21 12:12:52.510808257 +0000 UTC m=+0.373128283 container init 5c46e23442ad143a3478403b4f0198f9dbb92cbba610a4bbd5384bf49cb58f7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T12:12:52.989 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:52 smithi107 podman[149342]: 2024-09-21 12:12:52.515543419 +0000 UTC m=+0.377863437 container start 5c46e23442ad143a3478403b4f0198f9dbb92cbba610a4bbd5384bf49cb58f7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T12:12:52.990 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:52 smithi107 podman[149342]: 2024-09-21 12:12:52.585746925 +0000 UTC m=+0.448066952 container attach 5c46e23442ad143a3478403b4f0198f9dbb92cbba610a4bbd5384bf49cb58f7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T12:12:52.990 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 12:12:52 smithi107 podman[149342]: 2024-09-21 12:12:52.701961811 +0000 UTC m=+0.564281841 container died 5c46e23442ad143a3478403b4f0198f9dbb92cbba610a4bbd5384bf49cb58f7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-1-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:12:53.285 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.1.service' 2024-09-21T12:12:53.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:12:53.325 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T12:12:53.325 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T12:12:53.325 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.2 2024-09-21T12:12:53.637 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:53 smithi107 systemd[1]: Stopping Ceph osd.2 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:12:54.003 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:53 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T12:12:53.634+0000 7f3b81810640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:12:54.003 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:53 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T12:12:53.634+0000 7f3b81810640 -1 osd.2 768 *** Got signal Terminated *** 2024-09-21T12:12:54.003 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:53 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2[56704]: 2024-09-21T12:12:53.634+0000 7f3b81810640 -1 osd.2 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:12:59.464 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:59 smithi107 podman[149472]: 2024-09-21 12:12:59.188711015 +0000 UTC m=+5.647716085 container died acc5208321ec6ad493f192c9e1cbfb61399c1c038ab508c5b92247cba6d41bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:12:59.753 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:59 smithi107 podman[149472]: 2024-09-21 12:12:59.565844511 +0000 UTC m=+6.024849590 container cleanup acc5208321ec6ad493f192c9e1cbfb61399c1c038ab508c5b92247cba6d41bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T12:12:59.753 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:59 smithi107 bash[149472]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2 2024-09-21T12:13:00.131 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:12:59 smithi107 podman[149493]: 2024-09-21 12:12:59.816922374 +0000 UTC m=+0.626491788 container remove acc5208321ec6ad493f192c9e1cbfb61399c1c038ab508c5b92247cba6d41bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T12:13:00.387 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:00 smithi107 podman[149585]: 2024-09-21 12:13:00.032852915 +0000 UTC m=+0.016072644 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:13:00.387 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:00 smithi107 podman[149585]: 2024-09-21 12:13:00.142620544 +0000 UTC m=+0.125840306 container create efab32bb3f6925fabb62d95e71fc451727364c15fbe8ebbc22c6389ac0d93c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T12:13:00.388 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:00 smithi107 podman[149585]: 2024-09-21 12:13:00.386661164 +0000 UTC m=+0.369880887 container init efab32bb3f6925fabb62d95e71fc451727364c15fbe8ebbc22c6389ac0d93c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-21T12:13:00.753 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:00 smithi107 podman[149585]: 2024-09-21 12:13:00.391468935 +0000 UTC m=+0.374688655 container start efab32bb3f6925fabb62d95e71fc451727364c15fbe8ebbc22c6389ac0d93c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T12:13:00.753 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:00 smithi107 podman[149585]: 2024-09-21 12:13:00.478196269 +0000 UTC m=+0.461415995 container attach efab32bb3f6925fabb62d95e71fc451727364c15fbe8ebbc22c6389ac0d93c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T12:13:00.754 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:00 smithi107 podman[149585]: 2024-09-21 12:13:00.580584662 +0000 UTC m=+0.563804398 container died efab32bb3f6925fabb62d95e71fc451727364c15fbe8ebbc22c6389ac0d93c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T12:13:01.123 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 21 12:13:01 smithi107 podman[149628]: 2024-09-21 12:13:01.106050023 +0000 UTC m=+0.516010370 container remove efab32bb3f6925fabb62d95e71fc451727364c15fbe8ebbc22c6389ac0d93c44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:13:01.129 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.2.service' 2024-09-21T12:13:01.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:01.167 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T12:13:01.167 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T12:13:01.167 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.3 2024-09-21T12:13:01.487 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:01 smithi107 systemd[1]: Stopping Ceph osd.3 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:01.753 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:01 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T12:13:01.485+0000 7f47f01a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:13:01.753 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:01 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T12:13:01.485+0000 7f47f01a7640 -1 osd.3 768 *** Got signal Terminated *** 2024-09-21T12:13:01.753 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:01 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T12:13:01.485+0000 7f47f01a7640 -1 osd.3 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:13:03.852 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:03 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:03.575+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:04.503 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:04 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T12:13:04.169+0000 7f47e87b4640 -1 osd.3 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.369326+0000 front 2024-09-21T12:12:42.369373+0000 (oldest deadline 2024-09-21T12:13:03.369840+0000) 2024-09-21T12:13:04.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:04 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:04.532+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:05.503 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:05 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T12:13:05.189+0000 7f47e87b4640 -1 osd.3 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.369326+0000 front 2024-09-21T12:12:42.369373+0000 (oldest deadline 2024-09-21T12:13:03.369840+0000) 2024-09-21T12:13:05.523 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:05 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:05.067+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:05.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:05 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:05.521+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:06.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:06.044+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:06.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:05 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:05.894+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:06.503 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:06 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3[63075]: 2024-09-21T12:13:06.165+0000 7f47e87b4640 -1 osd.3 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.369326+0000 front 2024-09-21T12:12:42.369373+0000 (oldest deadline 2024-09-21T12:13:03.369840+0000) 2024-09-21T12:13:06.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:06.517+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:07.160 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:06 smithi107 podman[149712]: 2024-09-21 12:13:06.882189064 +0000 UTC m=+5.493260561 container died 0bf96ec90a4a675d02bafb8667393873c66863bddc90e204473eb43ed354caf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T12:13:07.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:06 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:06.875+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:07.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:07 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:07.055+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:07.501 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:07 smithi107 podman[149712]: 2024-09-21 12:13:07.241017386 +0000 UTC m=+5.852088926 container cleanup 0bf96ec90a4a675d02bafb8667393873c66863bddc90e204473eb43ed354caf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux ) 2024-09-21T12:13:07.501 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:07 smithi107 bash[149712]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3 2024-09-21T12:13:07.753 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:07 smithi107 podman[149733]: 2024-09-21 12:13:07.500308645 +0000 UTC m=+0.615792305 container remove 0bf96ec90a4a675d02bafb8667393873c66863bddc90e204473eb43ed354caf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T12:13:07.800 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:07 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:07.579+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:07.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:07 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:07.495+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:08.137 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:07 smithi107 podman[149826]: 2024-09-21 12:13:07.7571899 +0000 UTC m=+0.016018290 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:13:08.137 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:07 smithi107 podman[149826]: 2024-09-21 12:13:07.867861417 +0000 UTC m=+0.126689791 container create 6b0be35932175e715b3cc5eeb2fd49795684f34bc48fd8f4e3629849cb6c560a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T12:13:08.138 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:08 smithi107 podman[149826]: 2024-09-21 12:13:08.053448795 +0000 UTC m=+0.312277180 container init 6b0be35932175e715b3cc5eeb2fd49795684f34bc48fd8f4e3629849cb6c560a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T12:13:08.138 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:08 smithi107 podman[149826]: 2024-09-21 12:13:08.058429056 +0000 UTC m=+0.317257435 container start 6b0be35932175e715b3cc5eeb2fd49795684f34bc48fd8f4e3629849cb6c560a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T12:13:08.138 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:08 smithi107 podman[149826]: 2024-09-21 12:13:08.136518908 +0000 UTC m=+0.395347284 container attach 6b0be35932175e715b3cc5eeb2fd49795684f34bc48fd8f4e3629849cb6c560a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:13:08.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:08.062+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:08.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:07 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:07.904+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:08.503 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 21 12:13:08 smithi107 podman[149826]: 2024-09-21 12:13:08.246332836 +0000 UTC m=+0.505161225 container died 6b0be35932175e715b3cc5eeb2fd49795684f34bc48fd8f4e3629849cb6c560a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T12:13:08.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:08.542+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:08.800 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:08.568+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:08.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.3.service' 2024-09-21T12:13:08.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:08.840 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T12:13:08.840 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T12:13:08.840 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.4 2024-09-21T12:13:09.138 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:08 smithi193 systemd[1]: Stopping Ceph osd.4 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:09.138 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:08 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:08.898+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:09.138 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:09.012+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:09.543 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:09.163+0000 7f7986bdc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:13:09.543 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:09.163+0000 7f7986bdc640 -1 osd.4 768 *** Got signal Terminated *** 2024-09-21T12:13:09.543 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:09.163+0000 7f7986bdc640 -1 osd.4 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:13:09.543 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:09.540+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.137Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.138Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.138Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.139Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.141Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.142Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.544 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:09.142Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.107:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.107:8765: connect: connection refused" 2024-09-21T12:13:09.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:09.577+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:10.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:09 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:09.890+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:10.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:10 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:10.047+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:10.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:10 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:10.553+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:10.801 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:10 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:10.545+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:11.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:10 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:10.919+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:11.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:11.029+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:11.950 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:11.577+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:11.950 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:11.557+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:12.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:11.948+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:12.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:11.991+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:12.800 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:12 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:12.581+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:12.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:12 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:12.509+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:13.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:13 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:13.037+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:13.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:12 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:12.916+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:13.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:12 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:12.916+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:13.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:13 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:13.497+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:13.800 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:13 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4[37977]: 2024-09-21T12:13:13.589+0000 7f797f1e9640 -1 osd.4 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.571344+0000 front 2024-09-21T12:12:42.571267+0000 (oldest deadline 2024-09-21T12:13:06.671243+0000) 2024-09-21T12:13:14.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:14 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:14.063+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:14.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:13 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:13.923+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:14.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:13 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:13.923+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:14.791 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:14 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:14.453+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:14.791 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:14 smithi193 podman[68180]: 2024-09-21 12:13:14.494784054 +0000 UTC m=+5.455671498 container died 15621d9c505c4ab51843e67fba73f256e0b9fc7a7bf1d3ec24446d6a29f6a4d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:13:15.050 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:14 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:14.884+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:15.050 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:14 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:14.884+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:15.050 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:14 smithi193 podman[68180]: 2024-09-21 12:13:14.879780185 +0000 UTC m=+5.840667624 container cleanup 15621d9c505c4ab51843e67fba73f256e0b9fc7a7bf1d3ec24446d6a29f6a4d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux ) 2024-09-21T12:13:15.050 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:14 smithi193 bash[68180]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4 2024-09-21T12:13:15.474 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68201]: 2024-09-21 12:13:15.164638787 +0000 UTC m=+0.669500281 container remove 15621d9c505c4ab51843e67fba73f256e0b9fc7a7bf1d3ec24446d6a29f6a4d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T12:13:15.475 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:15 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:15.077+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:15.475 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:15 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:15.077+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:15.736 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:15 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:15.478+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:15.736 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:15 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:15.478+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:15.736 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68296]: 2024-09-21 12:13:15.376351874 +0000 UTC m=+0.016843954 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:13:15.736 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68296]: 2024-09-21 12:13:15.508467187 +0000 UTC m=+0.148959315 container create 5936b3bd32f60fad04a5f01eaf804231c0f2b72ce064a67471a1a1ae9537b6cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T12:13:15.737 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68296]: 2024-09-21 12:13:15.734970758 +0000 UTC m=+0.375462836 container init 5936b3bd32f60fad04a5f01eaf804231c0f2b72ce064a67471a1a1ae9537b6cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T12:13:16.040 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:15 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:15.862+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:16.040 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:15 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:15.862+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:16.041 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:16.038+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:16.041 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:16.038+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:16.041 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68296]: 2024-09-21 12:13:15.740488562 +0000 UTC m=+0.380980634 container start 5936b3bd32f60fad04a5f01eaf804231c0f2b72ce064a67471a1a1ae9537b6cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T12:13:16.041 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68296]: 2024-09-21 12:13:15.81818849 +0000 UTC m=+0.458680572 container attach 5936b3bd32f60fad04a5f01eaf804231c0f2b72ce064a67471a1a1ae9537b6cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-21T12:13:16.041 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 21 12:13:15 smithi193 podman[68296]: 2024-09-21 12:13:15.92801457 +0000 UTC m=+0.568506661 container died 5936b3bd32f60fad04a5f01eaf804231c0f2b72ce064a67471a1a1ae9537b6cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:13:16.558 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.4.service' 2024-09-21T12:13:16.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:16.599 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T12:13:16.599 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T12:13:16.599 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.5 2024-09-21T12:13:16.800 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:16 smithi193 systemd[1]: Stopping Ceph osd.5 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:16.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:16.507+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:16.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:16.507+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:17.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:16.886+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:17.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:16.886+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:17.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:16.935+0000 7f5911c97640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:13:17.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:16.935+0000 7f5911c97640 -1 osd.5 768 *** Got signal Terminated *** 2024-09-21T12:13:17.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:16 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:16.935+0000 7f5911c97640 -1 osd.5 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:13:17.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:17 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:17.077+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:17.301 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:17 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:17.077+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:17.363 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:17 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:17.012Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-21T12:13:17.363 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:17 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:17.013Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-21T12:13:17.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:17 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:17.510+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:17.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:17 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:17.510+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:18.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:18.111+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:18.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:18.111+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:18.301 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:17 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:17.935+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:18.301 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:17 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:17.935+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:18.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:18.470+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:18.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:18.470+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:19.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:19.083+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:19.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:19.083+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:19.301 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:18.985+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:19.301 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:18 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:18.985+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:19.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:19.464+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:19.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:19 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:19.464+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:20.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:20 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:20.066+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:20.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:20 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:20.066+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:20.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:20 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:20.020+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:20.301 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:20 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:20.020+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:20.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:20 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:20.440+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:20.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:20 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:20.440+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:21.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:21.042+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.367163+0000 front 2024-09-21T12:12:42.367147+0000 (oldest deadline 2024-09-21T12:13:04.667000+0000) 2024-09-21T12:13:21.300 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5[43659]: 2024-09-21T12:13:21.042+0000 7f590e2ac640 -1 osd.5 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:50.468499+0000 front 2024-09-21T12:12:50.468554+0000 (oldest deadline 2024-09-21T12:13:14.568476+0000) 2024-09-21T12:13:21.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:21.039+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:21.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:21.039+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:21.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:21.435+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:21.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:21.435+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:21.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:21 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:21.435+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:22.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:22 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:22.048+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:22.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:22 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:22.048+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:22.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:22 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:22.048+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:22.719 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:22 smithi193 podman[68426]: 2024-09-21 12:13:22.472571599 +0000 UTC m=+5.654564746 container died 14aaa4597efa5122f6d9efb544825095d44045a101489ce383ac7ea8519b337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T12:13:22.719 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:22 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:22.422+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:22.719 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:22 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:22.422+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:22.720 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:22 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:22.422+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:23.050 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:22 smithi193 podman[68426]: 2024-09-21 12:13:22.824718898 +0000 UTC m=+6.006712031 container cleanup 14aaa4597efa5122f6d9efb544825095d44045a101489ce383ac7ea8519b337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:13:23.050 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:22 smithi193 bash[68426]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5 2024-09-21T12:13:23.439 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68447]: 2024-09-21 12:13:23.142941385 +0000 UTC m=+0.663606688 container remove 14aaa4597efa5122f6d9efb544825095d44045a101489ce383ac7ea8519b337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True) 2024-09-21T12:13:23.439 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:23 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:23.075+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:23.439 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:23 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:23.075+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:23.439 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:23 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:23.075+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:23.706 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:23 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:23.437+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:23.706 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:23 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:23.437+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:23.706 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:23 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:23.437+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:23.706 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68541]: 2024-09-21 12:13:23.394292086 +0000 UTC m=+0.015879395 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:13:23.707 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68541]: 2024-09-21 12:13:23.493937601 +0000 UTC m=+0.115524945 container create 9fbad0c10502de5360282e6a5e6d1f6a255f2ed550b224c057c7174a951a77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T12:13:23.707 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68541]: 2024-09-21 12:13:23.705276723 +0000 UTC m=+0.326864034 container init 9fbad0c10502de5360282e6a5e6d1f6a255f2ed550b224c057c7174a951a77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-21T12:13:24.050 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68541]: 2024-09-21 12:13:23.710613465 +0000 UTC m=+0.332200770 container start 9fbad0c10502de5360282e6a5e6d1f6a255f2ed550b224c057c7174a951a77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T12:13:24.050 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68541]: 2024-09-21 12:13:23.804869261 +0000 UTC m=+0.426456576 container attach 9fbad0c10502de5360282e6a5e6d1f6a255f2ed550b224c057c7174a951a77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:13:24.051 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 12:13:23 smithi193 podman[68541]: 2024-09-21 12:13:23.902294849 +0000 UTC m=+0.523882156 container died 9fbad0c10502de5360282e6a5e6d1f6a255f2ed550b224c057c7174a951a77ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T12:13:24.451 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:24.083+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:24.451 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:24.083+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:24.451 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:24.083+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:24.478 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.5.service' 2024-09-21T12:13:24.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:24.516 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T12:13:24.517 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T12:13:24.517 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.6 2024-09-21T12:13:24.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:24.449+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:24.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:24.449+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:24.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:24.449+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:24.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 systemd[1]: Stopping Ceph osd.6 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:25.129 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:24.844+0000 7fb5b640a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:13:25.129 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:24.844+0000 7fb5b640a640 -1 osd.6 768 *** Got signal Terminated *** 2024-09-21T12:13:25.129 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:24 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:24.844+0000 7fb5b640a640 -1 osd.6 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:13:25.447 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:25 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:25.127+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:25.447 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:25 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:25.127+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:25.448 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:25 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:25.127+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:25.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:25 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:25.445+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:25.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:25 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:25.445+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:25.801 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:25 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:25.445+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:26.439 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:26.145+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:26.439 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:26.145+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:26.439 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:26.145+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:26.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:26.437+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:26.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:26.437+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:26.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:26 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:26.437+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:27.434 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:27 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:27.119+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:27.434 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:27 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:27.119+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:27.434 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:27 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:27.119+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:27.504 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:27 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:27.011Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-21T12:13:27.504 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:27 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:27.012Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-21T12:13:27.504 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:27 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:27.013Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-21T12:13:27.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:27 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:27.431+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:27.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:27 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:27.431+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:27.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:27 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:27.431+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:28.438 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:28.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:28.438 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:28.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:28.438 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:28.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:28.438 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:28.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:28.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:28.436+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:28.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:28.436+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:28.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:28 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:28.436+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:29.443 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:29.090+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:29.443 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:29.090+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:29.443 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:29.090+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:29.444 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:29.090+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:29.800 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:29.441+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.430658+0000 front 2024-09-21T12:12:42.430649+0000 (oldest deadline 2024-09-21T12:13:03.530449+0000) 2024-09-21T12:13:29.801 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:29.441+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:49.832497+0000 front 2024-09-21T12:12:49.832364+0000 (oldest deadline 2024-09-21T12:13:14.532416+0000) 2024-09-21T12:13:29.801 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:29 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6[49559]: 2024-09-21T12:13:29.441+0000 7fb5aea17640 -1 osd.6 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:58.633398+0000 front 2024-09-21T12:12:58.633373+0000 (oldest deadline 2024-09-21T12:13:20.933342+0000) 2024-09-21T12:13:30.374 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:30 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:30.110+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:30.374 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:30 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:30.110+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:30.375 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:30 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:30.110+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:30.375 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:30 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:30.110+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:30.661 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:30 smithi193 podman[68668]: 2024-09-21 12:13:30.373562387 +0000 UTC m=+5.631791298 container died 6679ef2ba058ad863be663fb42b21c541952e59d86fbb83eebaac4a2ff0e6f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T12:13:31.011 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:30 smithi193 podman[68668]: 2024-09-21 12:13:30.75898037 +0000 UTC m=+6.017209283 container cleanup 6679ef2ba058ad863be663fb42b21c541952e59d86fbb83eebaac4a2ff0e6f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T12:13:31.011 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:30 smithi193 bash[68668]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6 2024-09-21T12:13:31.300 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68690]: 2024-09-21 12:13:31.010337669 +0000 UTC m=+0.637968877 container remove 6679ef2ba058ad863be663fb42b21c541952e59d86fbb83eebaac4a2ff0e6f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:13:31.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:31 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:31.130+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:31.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:31 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:31.130+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:31.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:31 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:31.130+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:31.300 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:31 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:31.130+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:31.673 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68783]: 2024-09-21 12:13:31.27626391 +0000 UTC m=+0.016231447 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:13:31.673 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68783]: 2024-09-21 12:13:31.40288373 +0000 UTC m=+0.142851251 container create dfaeb84855ca7aa499b4412ebcaa227a16694fbe8044fd956d5b1ddd1e21668f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T12:13:31.673 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68783]: 2024-09-21 12:13:31.571927763 +0000 UTC m=+0.311895302 container init dfaeb84855ca7aa499b4412ebcaa227a16694fbe8044fd956d5b1ddd1e21668f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:13:31.673 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68783]: 2024-09-21 12:13:31.576917119 +0000 UTC m=+0.316884639 container start dfaeb84855ca7aa499b4412ebcaa227a16694fbe8044fd956d5b1ddd1e21668f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:13:31.673 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68783]: 2024-09-21 12:13:31.672027531 +0000 UTC m=+0.411995063 container attach dfaeb84855ca7aa499b4412ebcaa227a16694fbe8044fd956d5b1ddd1e21668f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T12:13:32.050 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 21 12:13:31 smithi193 podman[68783]: 2024-09-21 12:13:31.763863277 +0000 UTC m=+0.503830815 container died dfaeb84855ca7aa499b4412ebcaa227a16694fbe8044fd956d5b1ddd1e21668f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T12:13:32.318 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.103+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:32.318 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.104+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:32.318 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.104+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:32.318 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.104+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:32.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.6.service' 2024-09-21T12:13:32.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:32.381 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T12:13:32.381 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T12:13:32.381 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.7 2024-09-21T12:13:32.690 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 systemd[1]: Stopping Ceph osd.7 for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:33.050 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.688+0000 7f7a6265d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:13:33.050 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.688+0000 7f7a6265d640 -1 osd.7 768 *** Got signal Terminated *** 2024-09-21T12:13:33.050 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:32 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:32.688+0000 7f7a6265d640 -1 osd.7 768 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:13:33.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:33 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:33.100+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:33.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:33 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:33.100+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:33.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:33 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:33.100+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:33.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:33 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:33.100+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:34.551 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:34 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:34.116+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:34.551 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:34 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:34.116+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:34.551 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:34 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:34.116+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:34.551 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:34 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:34.116+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:35.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:35 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:35.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:35.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:35 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:35.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:35.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:35 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:35.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:35.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:35 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:35.084+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:36.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:36 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:36.105+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:36.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:36 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:36.105+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:36.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:36 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:36.105+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:36.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:36 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:36.105+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:36.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:36 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:36.105+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.193:6804 osd.4 since back 2024-09-21T12:13:10.159070+0000 front 2024-09-21T12:13:10.159031+0000 (oldest deadline 2024-09-21T12:13:36.059049+0000) 2024-09-21T12:13:37.503 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:37 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:37.012Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-21T12:13:37.503 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:37 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:37.013Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-21T12:13:37.503 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:37 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:37.013Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-21T12:13:37.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:37 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:37.123+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-21T12:12:42.755558+0000 front 2024-09-21T12:12:42.755373+0000 (oldest deadline 2024-09-21T12:13:05.655259+0000) 2024-09-21T12:13:37.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:37 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:37.123+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-21T12:12:45.655876+0000 front 2024-09-21T12:12:45.655694+0000 (oldest deadline 2024-09-21T12:13:12.056253+0000) 2024-09-21T12:13:37.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:37 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:37.123+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-21T12:12:57.357380+0000 front 2024-09-21T12:12:57.357297+0000 (oldest deadline 2024-09-21T12:13:21.457233+0000) 2024-09-21T12:13:37.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:37 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:37.123+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-21T12:13:01.457806+0000 front 2024-09-21T12:13:01.457779+0000 (oldest deadline 2024-09-21T12:13:27.357754+0000) 2024-09-21T12:13:37.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:37 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7[55429]: 2024-09-21T12:13:37.123+0000 7f7a5ac6a640 -1 osd.7 768 heartbeat_check: no reply from 172.21.15.193:6804 osd.4 since back 2024-09-21T12:13:10.159070+0000 front 2024-09-21T12:13:10.159031+0000 (oldest deadline 2024-09-21T12:13:36.059049+0000) 2024-09-21T12:13:38.288 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:38 smithi193 podman[68911]: 2024-09-21 12:13:38.01377263 +0000 UTC m=+5.413665155 container died 9e9f92ce2b025844a77988314ac843400d251786292fd014387d786fac822ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T12:13:38.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:38 smithi193 podman[68911]: 2024-09-21 12:13:38.377974283 +0000 UTC m=+5.777866863 container cleanup 9e9f92ce2b025844a77988314ac843400d251786292fd014387d786fac822ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T12:13:38.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:38 smithi193 bash[68911]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7 2024-09-21T12:13:38.968 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:38 smithi193 podman[68945]: 2024-09-21 12:13:38.65414905 +0000 UTC m=+0.633097686 container remove 9e9f92ce2b025844a77988314ac843400d251786292fd014387d786fac822ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T12:13:39.266 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:38 smithi193 podman[69042]: 2024-09-21 12:13:38.870353245 +0000 UTC m=+0.015827874 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:13:39.266 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:38 smithi193 podman[69042]: 2024-09-21 12:13:38.971649318 +0000 UTC m=+0.117123932 container create f02c15a28f959c6918bc5f0123a59c8bd5a886d56e136ebb52acc28736bd2fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T12:13:39.267 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:39 smithi193 podman[69042]: 2024-09-21 12:13:39.174016726 +0000 UTC m=+0.319491356 container init f02c15a28f959c6918bc5f0123a59c8bd5a886d56e136ebb52acc28736bd2fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7-deactivate, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T12:13:39.267 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:39 smithi193 podman[69042]: 2024-09-21 12:13:39.179007338 +0000 UTC m=+0.324481952 container start f02c15a28f959c6918bc5f0123a59c8bd5a886d56e136ebb52acc28736bd2fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:13:39.267 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:39 smithi193 podman[69042]: 2024-09-21 12:13:39.26542236 +0000 UTC m=+0.410896973 container attach f02c15a28f959c6918bc5f0123a59c8bd5a886d56e136ebb52acc28736bd2fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T12:13:39.550 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 21 12:13:39 smithi193 podman[69042]: 2024-09-21 12:13:39.374110455 +0000 UTC m=+0.519585077 container died f02c15a28f959c6918bc5f0123a59c8bd5a886d56e136ebb52acc28736bd2fe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:13:39.953 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@osd.7.service' 2024-09-21T12:13:39.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:39.996 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T12:13:39.996 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-21T12:13:39.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@rgw.foo.a 2024-09-21T12:13:40.299 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:40 smithi107 systemd[1]: Stopping Ceph rgw.foo.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:40.753 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:40 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a[67939]: 2024-09-21T12:13:40.297+0000 7fb697aee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:13:40.753 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:40 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a[67939]: 2024-09-21T12:13:40.297+0000 7fb69b35a880 -1 shutting down 2024-09-21T12:13:47.503 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:47.013Z caller=dispatch.go:353 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-21T12:13:47.503 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:47.013Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-21T12:13:47.504 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:47 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:47.014Z caller=notify.go:848 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] aggrGroup="{}/{}:{alertname=\"CephNodeDiskspaceWarning\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-21T12:13:50.656 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:50 smithi107 bash[149966]: time="2024-09-21T12:13:50Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-21T12:13:50.656 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:50 smithi107 podman[149966]: 2024-09-21 12:13:50.317036173 +0000 UTC m=+10.124129363 container died a24de966ed5b0fc31b2b5652635abf7e7b29d5c1487f49bb53d82080d0d8aec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:13:50.931 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:50 smithi107 podman[149966]: 2024-09-21 12:13:50.76210408 +0000 UTC m=+10.569197197 container cleanup a24de966ed5b0fc31b2b5652635abf7e7b29d5c1487f49bb53d82080d0d8aec3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T12:13:50.931 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 21 12:13:50 smithi107 bash[149966]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-rgw-foo-a 2024-09-21T12:13:51.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@rgw.foo.a.service' 2024-09-21T12:13:51.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:51.188 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-21T12:13:51.188 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-21T12:13:51.188 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1795c04c-780f-11ef-baf6-efdc52797490@prometheus.a 2024-09-21T12:13:51.482 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 systemd[1]: Stopping Ceph prometheus.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-21T12:13:51.757 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.481Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-21T12:13:51.758 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.482Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-21T12:13:51.758 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.482Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-21T12:13:51.758 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a[64531]: ts=2024-09-21T12:13:51.482Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-21T12:13:51.758 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 podman[69171]: 2024-09-21 12:13:51.491701931 +0000 UTC m=+0.105725098 container died 7924e77b724b4fcad71cce2a1ca2870402622af56d6103c9dabf24eecd05cfd5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:52.050 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 podman[69171]: 2024-09-21 12:13:51.862103837 +0000 UTC m=+0.476127001 container cleanup 7924e77b724b4fcad71cce2a1ca2870402622af56d6103c9dabf24eecd05cfd5 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:52.050 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 21 12:13:51 smithi193 bash[69171]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-prometheus-a 2024-09-21T12:13:52.280 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1795c04c-780f-11ef-baf6-efdc52797490@prometheus.a.service' 2024-09-21T12:13:52.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:13:52.315 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-21T12:13:52.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1795c04c-780f-11ef-baf6-efdc52797490 --force --keep-logs 2024-09-21T12:13:52.518 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T12:13:54.377 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 systemd[1]: Stopping Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:54.378 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a[76749]: ts=2024-09-21T12:13:54.335Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-21T12:13:54.692 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 podman[150465]: 2024-09-21 12:13:54.376696935 +0000 UTC m=+0.138896772 container died cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:54.692 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 podman[150465]: 2024-09-21 12:13:54.691682171 +0000 UTC m=+0.453881997 container cleanup cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:54.986 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 bash[150465]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a 2024-09-21T12:13:54.986 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 podman[150482]: 2024-09-21 12:13:54.89283228 +0000 UTC m=+0.512537395 container remove cdd8db53662cfcca8a12cf6412ffb6ad9bd8aaab2d74d085be900983b602a974 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:54.986 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:54 smithi107 podman[150482]: 2024-09-21 12:13:54.985316891 +0000 UTC m=+0.605022006 volume remove c50b50c27cd2424f744f702ca4c324d099e6326394c6156208b6925367f0e47e 2024-09-21T12:13:55.253 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:55 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-21T12:13:55.253 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:55 smithi107 systemd[1]: Stopped Ceph alertmanager.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T12:13:55.253 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 21 12:13:55 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@alertmanager.a.service: Consumed 1.764s CPU time. 2024-09-21T12:13:55.560 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:55 smithi107 systemd[1]: Stopping Ceph node-exporter.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:55.856 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:55 smithi107 podman[150603]: 2024-09-21 12:13:55.559332936 +0000 UTC m=+0.118856587 container died 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:56.181 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:55 smithi107 podman[150603]: 2024-09-21 12:13:55.946268923 +0000 UTC m=+0.505792563 container cleanup 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:56.181 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:55 smithi107 bash[150603]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a 2024-09-21T12:13:56.474 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:56 smithi107 podman[150619]: 2024-09-21 12:13:56.18032957 +0000 UTC m=+0.620665129 container remove 33f65977667cc66a92aa14f3cabe05dda6bb466f64699e0081f7c51be746b16d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T12:13:56.474 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:56 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-21T12:13:56.474 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:56 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-21T12:13:56.474 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:56 smithi107 systemd[1]: Stopped Ceph node-exporter.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T12:13:56.474 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 21 12:13:56 smithi107 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.a.service: Consumed 3.619s CPU time. 2024-09-21T12:13:56.916 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1795c04c-780f-11ef-baf6-efdc52797490 --force --keep-logs 2024-09-21T12:13:57.118 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T12:13:58.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:13:58 smithi193 systemd[1]: Stopping Ceph iscsi.iscsi.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:13:58.800 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:13:58 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a[60634]: debug Shutdown received 2024-09-21T12:14:08.900 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:08 smithi193 bash[69589]: time="2024-09-21T12:14:08Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-21T12:14:08.900 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:08 smithi193 podman[69589]: 2024-09-21 12:14:08.61867996 +0000 UTC m=+10.112217357 container died 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T12:14:09.191 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 podman[69589]: 2024-09-21 12:14:09.006636166 +0000 UTC m=+10.500173542 container cleanup 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:14:09.192 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 bash[69589]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a 2024-09-21T12:14:09.447 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 podman[69612]: 2024-09-21 12:14:09.291978084 +0000 UTC m=+0.673798936 container remove 416514cc423a1221e5ddbac700920a65883e2d6d85da6051d7b2981fda1669ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-iscsi-iscsi-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2) 2024-09-21T12:14:09.448 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-21T12:14:09.767 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-21T12:14:09.768 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 systemd[1]: Stopped Ceph iscsi.iscsi.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T12:14:09.768 INFO:journalctl@ceph.iscsi.iscsi.a.smithi193.stdout:Sep 21 12:14:09 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@iscsi.iscsi.a.service: Consumed 2.755s CPU time. 2024-09-21T12:14:11.050 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:10 smithi193 systemd[1]: Stopping Ceph grafana.a for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:14:11.400 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=server t=2024-09-21T12:14:11.143222293Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-21T12:14:11.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=tracing t=2024-09-21T12:14:11.143622465Z level=info msg="Closing tracing" 2024-09-21T12:14:11.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=ticker t=2024-09-21T12:14:11.143642548Z level=info msg=stopped last_tick=2024-09-21T12:14:10Z 2024-09-21T12:14:11.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a[63246]: logger=grafana-apiserver t=2024-09-21T12:14:11.143659117Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-21T12:14:11.401 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 podman[69895]: 2024-09-21 12:14:11.153632278 +0000 UTC m=+0.104583433 container died 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 (image=quay.io/ceph/grafana:10.4.8, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T12:14:11.677 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 podman[69895]: 2024-09-21 12:14:11.474130452 +0000 UTC m=+0.425081660 container cleanup 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 (image=quay.io/ceph/grafana:10.4.8, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T12:14:11.677 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 bash[69895]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a 2024-09-21T12:14:12.050 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 podman[69914]: 2024-09-21 12:14:11.675523095 +0000 UTC m=+0.523033731 container remove 7084c873bda9957d2b402ee15c0a0978eb4ec1468b1f0f08b72e74478407fcc5 (image=quay.io/ceph/grafana:10.4.8, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T12:14:12.050 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@grafana.a.service: Deactivated successfully. 2024-09-21T12:14:12.050 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 systemd[1]: Stopped Ceph grafana.a for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T12:14:12.050 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 21 12:14:11 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@grafana.a.service: Consumed 12.999s CPU time. 2024-09-21T12:14:12.350 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:12 smithi193 systemd[1]: Stopping Ceph node-exporter.b for 1795c04c-780f-11ef-baf6-efdc52797490... 2024-09-21T12:14:12.613 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:12 smithi193 podman[70031]: 2024-09-21 12:14:12.349249694 +0000 UTC m=+0.117194873 container died 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T12:14:12.963 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:12 smithi193 podman[70031]: 2024-09-21 12:14:12.711410785 +0000 UTC m=+0.479355956 container cleanup 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T12:14:12.963 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:12 smithi193 bash[70031]: ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b 2024-09-21T12:14:13.273 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:12 smithi193 podman[70047]: 2024-09-21 12:14:12.962467181 +0000 UTC m=+0.616849262 container remove 7a90ea697b8e105a9f79152e278b370d0a17a8bb7cfe00ed46a59a4de2182352 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-1795c04c-780f-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T12:14:13.274 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:12 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-21T12:14:13.274 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:13 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-21T12:14:13.274 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:13 smithi193 systemd[1]: Stopped Ceph node-exporter.b for 1795c04c-780f-11ef-baf6-efdc52797490. 2024-09-21T12:14:13.274 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 21 12:14:13 smithi193 systemd[1]: ceph-1795c04c-780f-11ef-baf6-efdc52797490@node-exporter.b.service: Consumed 3.615s CPU time. 2024-09-21T12:14:13.759 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:14:13.792 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:14:13.819 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T12:14:13.821 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877/remote/smithi107/crash 2024-09-21T12:14:13.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/crash -- . 2024-09-21T12:14:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T12:14:13.879 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-09-21T12:14:13.881 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877/remote/smithi193/crash 2024-09-21T12:14:13.881 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/crash -- . 2024-09-21T12:14:13.912 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/1795c04c-780f-11ef-baf6-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T12:14:13.912 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-21T12:14:13.913 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T12:14:13.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v CEPHADM_STRAY_DAEMON | head -n 1 2024-09-21T12:14:13.948 INFO:tasks.cephadm:Compressing logs... 2024-09-21T12:14:13.948 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T12:14:13.991 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T12:14:14.015 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.0.log 2024-09-21T12:14:14.015 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-09-21T12:14:14.016 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.1.log 2024-09-21T12:14:14.016 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-volume.log 2024-09-21T12:14:14.016 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.0.log: /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.audit.log 2024-09-21T12:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.c.log 2024-09-21T12:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.cephadm.log 2024-09-21T12:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.audit.log: /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.a.log 2024-09-21T12:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.2.log 2024-09-21T12:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.a.log: 89.2%gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mgr.y.log 2024-09-21T12:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr: -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.cephadm.log.gz 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/tcmu-runner.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.7.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-volume.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr: 63.3% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/tcmu-runner.log.gz 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.4.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.audit.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.cephadm.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.6.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.5.log 2024-09-21T12:14:14.021 INFO:teuthology.orchestra.run.smithi193.stderr: 79.8%/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.6.log: -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.cephadm.log.gz 2024-09-21T12:14:14.022 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log 2024-09-21T12:14:14.022 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mgr.x.log 2024-09-21T12:14:14.023 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log: /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mgr.x.log: 86.9% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log.gz 2024-09-21T12:14:14.023 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.b.log 2024-09-21T12:14:14.023 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T12:14:14.027 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.audit.log.gz 2024-09-21T12:14:14.029 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: 91.6% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T12:14:14.031 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log 2024-09-21T12:14:14.034 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mgr.y.log: 95.1% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.audit.log.gz 2024-09-21T12:14:14.034 INFO:teuthology.orchestra.run.smithi193.stderr: 91.2% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-volume.log.gz 2024-09-21T12:14:14.035 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-volume.log.gz 2024-09-21T12:14:14.036 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-client.rgw.foo.a.log 2024-09-21T12:14:14.036 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.3.log 2024-09-21T12:14:14.038 INFO:teuthology.orchestra.run.smithi193.stderr: 91.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T12:14:14.043 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-client.rgw.foo.a.log: 92.7% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph.log.gz 2024-09-21T12:14:14.050 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T12:14:14.072 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T12:14:14.090 INFO:teuthology.orchestra.run.smithi107.stderr: 94.1% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-client.rgw.foo.a.log.gz 2024-09-21T12:14:14.352 INFO:teuthology.orchestra.run.smithi107.stderr: 91.3% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T12:14:15.276 INFO:teuthology.orchestra.run.smithi193.stderr: 92.9% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.b.log.gz 2024-09-21T12:14:15.364 INFO:teuthology.orchestra.run.smithi107.stderr: 92.6% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.c.log.gz 2024-09-21T12:14:16.208 INFO:teuthology.orchestra.run.smithi107.stderr: 92.2% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-mon.a.log.gz 2024-09-21T12:14:20.121 INFO:teuthology.orchestra.run.smithi193.stderr: 93.8% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.6.log.gz 2024-09-21T12:14:20.248 INFO:teuthology.orchestra.run.smithi107.stderr: 93.9% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.0.log.gz 2024-09-21T12:14:20.270 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.5.log.gz 2024-09-21T12:14:20.790 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.7.log.gz 2024-09-21T12:14:20.977 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.1.log.gz 2024-09-21T12:14:21.185 INFO:teuthology.orchestra.run.smithi107.stderr: 93.9% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.2.log.gz 2024-09-21T12:14:21.254 INFO:teuthology.orchestra.run.smithi193.stderr: 94.0% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.4.log.gz 2024-09-21T12:14:21.258 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-09-21T12:14:21.258 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m7.250s 2024-09-21T12:14:21.258 INFO:teuthology.orchestra.run.smithi193.stderr:user 0m26.684s 2024-09-21T12:14:21.258 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m0.961s 2024-09-21T12:14:21.609 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/1795c04c-780f-11ef-baf6-efdc52797490/ceph-osd.3.log.gz 2024-09-21T12:14:21.611 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-21T12:14:21.611 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m7.607s 2024-09-21T12:14:21.611 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m30.786s 2024-09-21T12:14:21.611 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m1.054s 2024-09-21T12:14:21.612 INFO:tasks.cephadm:Archiving logs... 2024-09-21T12:14:21.612 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877/remote/smithi107/log 2024-09-21T12:14:21.613 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T12:14:25.582 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877/remote/smithi193/log 2024-09-21T12:14:25.583 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T12:14:29.171 INFO:tasks.cephadm:Removing cluster... 2024-09-21T12:14:29.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1795c04c-780f-11ef-baf6-efdc52797490 --force 2024-09-21T12:14:29.378 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T12:14:29.660 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1795c04c-780f-11ef-baf6-efdc52797490 --force 2024-09-21T12:14:29.864 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 1795c04c-780f-11ef-baf6-efdc52797490 2024-09-21T12:14:30.153 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T12:14:30.153 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T12:14:30.170 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T12:14:30.185 INFO:tasks.cephadm:Teardown complete 2024-09-21T12:14:30.185 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T12:14:30.198 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T12:14:30.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T12:14:30.212 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T12:14:30.300 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T12:14:30.308 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T12:14:30.308 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T12:14:30.311 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T12:14:30.323 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-21T12:14:30.326 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-21T12:14:30.328 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T12:14:30.328 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-21T12:14:30.328 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 -132us[ -129us] +/- 59ms 2024-09-21T12:14:30.328 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 +94us[ +98us] +/- 71ms 2024-09-21T12:14:30.328 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +107us[ +110us] +/- 71ms 2024-09-21T12:14:30.328 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T12:14:30.330 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T12:14:30.330 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-21T12:14:30.330 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 -150us[ -150us] +/- 59ms 2024-09-21T12:14:30.330 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 +90us[ +90us] +/- 71ms 2024-09-21T12:14:30.330 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +115us[ +115us] +/- 71ms 2024-09-21T12:14:30.331 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T12:14:30.331 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T12:14:30.340 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T12:14:30.341 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T12:14:30.348 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T12:14:30.443 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T12:14:30.512 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T12:14:30.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:30.540 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T12:14:30.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:30.569 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T12:14:30.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T12:14:30.587 INFO:teuthology.task.internal:Duration was 2074.593094 seconds 2024-09-21T12:14:30.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T12:14:30.595 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T12:14:30.595 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T12:14:30.598 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T12:14:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T12:14:30.648 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T12:14:31.085 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T12:14:31.086 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T12:14:31.086 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T12:14:31.106 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T12:14:31.107 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T12:14:31.152 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T12:14:31.153 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T12:14:31.155 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T12:14:31.219 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T12:14:31.219 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T12:14:31.222 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T12:14:31.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T12:14:31.647 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T12:14:31.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T12:14:31.672 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T12:14:31.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T12:14:31.707 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:31.714 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:31.735 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-21T12:14:31.763 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-21T12:14:31.784 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:31.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:31.803 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:31.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:31.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T12:14:31.853 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T12:14:31.854 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877/remote/smithi107 2024-09-21T12:14:31.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T12:14:31.915 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913877/remote/smithi193 2024-09-21T12:14:31.916 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T12:14:31.951 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T12:14:31.951 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T12:14:31.954 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T12:14:32.000 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T12:14:32.012 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T12:14:32.012 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T12:14:32.027 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T12:14:32.027 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T12:14:32.030 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T12:14:32.042 INFO:teuthology.orchestra.run.smithi107.stdout: 265613 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 12:14 /home/ubuntu/cephtest 2024-09-21T12:14:32.055 INFO:teuthology.orchestra.run.smithi193.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 12:14 /home/ubuntu/cephtest 2024-09-21T12:14:32.056 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T12:14:32.065 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T12:14:32.084 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} duration: 2074.5930943489075 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-21T12:14:32.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T12:14:32.162 INFO:teuthology.run:pass